IP Library › Granted Patent US 10,715,816
Granted Patent B2
US 10,715,816 · App. 15/349,989 · Granted Jul 14, 2020

Adaptive chroma downsampling and color space conversion techniques

Inventors: Alexandros Tourapis (Milpitas, CA); David W. Singer (San Francisco, CA); Hsi-Jung Wu (San Jose, CA); Yeping Su (Sunnyvale, CA)
Assignee: Apple Inc.
H04N19/154H04N1/646H04N9/646H04N9/67H04N19/105H04N19/124H04N19/13H04N19/15H04N19/51H04N19/61H04N19/17H04N19/186
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,715,816
App. No.
15/349,989
Granted
Jul 14, 2020
Kind
B2
Abstract

Techniques for selecting a luminance value for color space conversion are disclosed. Techniques include determining values for Cb and Cr from values for R′, G′, and B′; producing a reconstructed Cb* value and a reconstructed Cr* value by processing the Cb and Cr values; and determining a plurality of Y′ value options from the values for Cb* and Cr*. A Y′ output value may be selected based on the plurality of Y′ value options.

Claims (108)

1. An image conversion method, comprising:

determining values for Cb and Cr for image data according to a transform from the image data's source color space to a Y′CrCb color space;

producing a reconstructed Cb* value and a reconstructed Cr* value by processing the Cb and Cr values according to a lossy process, then inverting the lossy process;

determining a plurality of candidate Y′ values from the image data in the source color space and the Cb* and Cr* values;

deriving a final Y′ value from the plurality of candidate Y′ values; and

outputting the derived final Y′ value, Cb value, and Cr value as converted image data.

2. The method of claim 1 , wherein the source color space is R′G′B′ and the deriving comprises:

determining a first Y′ value (Y Cb ′) from B′ and Cb*;

determining a second Y′ value (Y Cr ′) from R′ and Cr*;

determining a third Y′ value (Y) from Cb*, Cr* and G′, and the deriving includes blending the Y Cb ′, Y Cr ′, and Y G ′ values.

3. The method of claim 1 , wherein the source color space is R′G′B′ and the deriving comprises:

determining a first Y′ value (Y Cb ′) from B′ and Cb*;

determining a second Y′ value (Y Cr ′) from R′ and Cr*;

determining a third Y′ value (Y G ′) from Cb*, Cr* and G′;

calculating distortion metrics corresponding to the first, second, and third Y′ values, where the distortion metric is based on the Cb* and Cr* values and the corresponding Y′ value; and

the deriving includes selecting the final Y′ from amongst the first, second, and third Y′ values with a lowest corresponding distortion.

4. The method of claim 1 , further comprising:

determining a minimum Y′ value option and a maximum Y′ value option based on Cb* and Cr*;

searching for a minimum of a distortion metric over Y′ values that range from the minimum Y′ value option to the maximum Y′ value option; and

the deriving includes selecting the final Y′ value corresponding to the minimum of the distortion metric.

5. The method of claim 1 , wherein:

the source color space is R′G′B′,

the deriving is based on a distortion metric for candidate Y′ values; and

the distortion metric for a candidate Y′ value is calculated as a weighted linear combination of distortions values for R′, G′, and B′ where:

the distortion value for R′ is based on Cr and the candidate Y′ value;

the distortion value for B′ is based on Cb and the candidate Y′ value; and

the distortion value for G′ is based on G and the candidate Y′ value.

6. The method of claim 1 , wherein the reconstructed Cb* and Cr* values are produced by one or more of the following:

quantizing and then inverse quantizing the Cb and Cr values;

downscaling and then upscaling the Cb and Cr values; and

encoding and then decoding the Cb and Cr values.

7. The method of claim 1 , wherein the deriving includes a micrograding process.

8. A non-transitory computer readable medium comprising instructions for color space conversion, the medium comprising instructions that, when executed on a computer, cause at least:

determining values for Cb and Cr for image data according to a transform from the image data's source color space to a Y′CrCb color space;

producing a reconstructed Cb* value and a reconstructed Cr* value by processing the Cb and Cr values according to a lossy process, then inverting the lossy process;

determining a plurality of candidate Y′ values by combining the image data in the source color space with the Cb* and Cr* values;

deriving a final Y′ value from the plurality of candidate Y′ values'; and

outputting the derived final Y′ value, Cb value, and Cr value as converted image data.

9. The non-transitory computer readable medium of claim 8 , wherein the source color space is R′G′B′ and wherein the instructions for deriving a Y′ value include instructions to cause:

determining a first Y′ value (Y Cb ′) from B′ and Cb*;

determining a second Y′ value (Y Cr ′) from R′ and Cr*;

determining a third Y′ value (Y G ′) from Cb*, Cr* and G′, and

blending the Y Cb ′, Y Cr ′ and Y G ′ values.

10. The non-transitory computer readable medium of claim 8 , wherein the source color space is R′G′B′ and wherein the instructions for deriving a Y′ value include instructions to cause:

determining a first Y′ value (Y Cb ′) from B′ and Cb*;

determining a second Y′ value (Y Cr ′) from R′ and Cr*;

determining a third Y′ value (Y G ′) from Cb*, Cr* and G′, and

calculating distortion metrics corresponding to the first, second, and third Y′ values, where the distortion metric is based on the Cb* and Cr* values and the corresponding Y′ value, and where the distortion metrics are computed as the square error difference between the original luminance Y value and the reconstructed luminance value for each Y′, Cb*, and Cr* combination; and

selecting the final Y′ from amongst the first, second, and third Y′ values with the lowest corresponding distortion.

11. The non-transitory computer readable medium of claim 8 , wherein the instructions further cause:

determining a minimum Y′ value option and a maximum Y′ value option based on Cb* and Cr*;

searching for a minimum of a distortion metric over Y′ values that range from the minimum Y′ value option to the maximum Y′ value option; and

the deriving includes selecting the final Y′ value corresponding to the minimum of the distortion metric.

12. The non-transitory computer readable medium of claim 8 , wherein the instructions further cause:

the source color space is R′G′B′,

the deriving is based on a distortion metric for candidate Y′ values; and

the distortion metric for a candidate Y′ value is calculated as a weighted linear combination of distortions values for R′, G′, and B′ where:

the distortion value for R′ is based on Cr and the candidate Y′ value;

the distortion value for B′ is based on Cb and the candidate Y′ value; and

the distortion value for G′ is based on G and the candidate Y′ value.

13. The non-transitory computer readable medium of claim 8 , wherein the instructions further cause:

quantizing and then inverse quantizing the Cb and Cr values;

downscaling and then upscaling the Cb and Cr values; and

encoding and then decoding the Cb and Cr values.

14. The non-transitory computer readable medium of claim 8 , wherein the deriving includes a micrograding process.

15. A system comprising at least a processor and a memory storing instructions thereon that, when executed by the system, cause at least:

determining values for Cb and Cr for image data according to a transform from the image data's source color space to a Y′CrCb color space;

producing a reconstructed Cb* value and a reconstructed Cr* value by processing the Cb and Cr values according to a lossy process, then inverting the lossy process;

determining a plurality of candidate Y′ values by combining the image data in the source color space with the Cb* and Cr* values;

deriving a final Y′ value from the plurality of candidate Y′ values'; and

outputting the derived final Y′ value, Cb value, and Cr value as converted image data.

16. The system of claim 15 , wherein the instructions for the determining includes instructions that cause:

determining a first Y′ value (Y Cb ′) from B′ and Cb*;

determining a second Y′ value (Y Cr ′) from R′ and Cr*;

determining a third Y′ value (Y G ′) from Cb*, Cr* and G′, and

the deriving includes blending the Y Cb ′, Y Cr ′ and Y G ′ values.

17. The system of claim 15 , wherein the instructions for the determining includes instructions that cause:

determining a first Y′ value (Y Cb ′) from B′ and Cb*;

determining a second Y′ value (Y Cr ′) from R′ and Cr*;

determining a third Y′ value (Y G ′) from Cb*, Cr* and G′;

calculating distortion metrics corresponding to the first, second, and third Y′ values, where the distortion metric is based on the Cb* and Cr* values and the corresponding Y′ value, and where the distortion metrics are computed as the square error difference between the original luminance Y value and the reconstructed luminance value for each Y′, Cb*, and Cr* combination; and

the deriving includes selecting the final Y′ from amongst the first, second, and third Y′ values with the lowest corresponding distortion.

18. The system of claim 15 , wherein the instructions further cause:

determining a minimum Y′ value option and a maximum Y′ value option based on Cb* and Cr*;

searching for a minimum of a distortion metric over Y′ values that range from the minimum Y′ value option to the maximum Y′ value option; and

the deriving includes selecting the Y′ value corresponding to the minimum of the distortion metric.

19. The system of claim 15 , wherein

the source color space is R′G′B′,

the deriving is based on a distortion metric for candidate Y′ values; and

the distortion metric for a candidate Y′ value is calculated as a weighted linear combination of distortions values for R′, G′, and B′ where:

the distortion value for R′ is based on Cr and the candidate Y′ value;

the distortion value for B′ is based on Cb and the candidate Y′ value; and

the distortion value for G′ is based on G and the candidate Y′ value.

20. The system of claim 15 , wherein the instructions further cause:

quantizing and then inverse quantizing the Cb and Cr values;

downscaling and then upscaling the Cb and Cr values; and

encoding and then decoding the Cb and Cr values.

21. The method of claim 3 , wherein the distortion is computed as the square error difference between the original luminance Y value and the reconstructed luminance value for each Y′, Cb*, and Cr* combination.

22. This method of claim 1 , further comprising:

encoding the derived final Y′ value, the Cb value, and the Cr value into an encoded video stream.

23. An image conversion method, comprising:

determining values for Cb and Cr color components from source image data in an RGB color space including R′, G′ and B′ color components according to a transform from the RGB color space to a Y′CrCb color space;

producing a reconstructed Cb* value and a reconstructed Cr* value by processing the Cb and Cr values according to a lossy process, then inverting the lossy process;

determining a first Y′ value from B′ and Cb*;

determining a second Y′ value from R′ and Cr*;

determining a third Y′ value from Cb*, Cr* and G′, and

deriving a final Y′ value by blending the first, second, and third Y′ values; and

outputting the final Y′ value, Cb value, and Cr value as converted image data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 13, 2017
From: TOURAPIS, ALEXANDROS; SINGER, DAVID W.; WU, HSI-JUNG; SU, YEPING
To: APPLE INC.
Reel/Frame 041235/0874 →
Continuity (2)
Provisional Application 62254101 · Nov 11, 2015
Related Publication 20170134731A1 · May 11, 2017
Cited By (1)
US 12,407,944