Image processing method, image processing device, semiconductor device, electronic apparatus, image processing program, and computer-readable storage medium
View Patent ↗An image processing method comprising a step of performing a color interpolation process on image data output from a single-chip color image capturing sensor to calculate color data for each pixel and a step of converting the color data using a predetermined transformation formula and of filtering the converted data, and the transformation formula is replaced based on the magnitude of a color value of the color data. When a color value of a specific color is large, the transformation formula can be appropriately changed. Therefore, it is possible to prevent the lowering of the overall resolution without being affected by the resolution of the color. Further, it is possible to reliably reproduce a smooth and natural edge of an image without adding other processes to the color interpolation process.
1. A method of processing an image performed by an image processing device, the image being output from a single-chip color image capturing sensor having a first color pixel, a second color pixel, and a third color pixel, the method comprising:
calculating, with a processor of the image processing device, first color data including a first color value, a second color value, and a third color value by an interpolation process, the first color data being color data of the first color pixel;
converting, with the processor, the first color data to a first brightness signal Y and a first color-difference signal C, the first brightness signal Y being calculated by a first transformation formula selected from a plurality of transformation formulas based on a threshold value condition determined by a difference between the first color value and a value proportionate to the second color value and by a difference between the third color value and the value proportionate to the second color value, and the first color-difference signal C calculated by a difference between the first color value and the first brightness signal Y, the converting of the first color data including converting the first color data into a second brightness signal Y and a second color-difference signal C, the second brightness signal Y being calculated by a second transformation formula selected from the plurality of transformation formulas based on the threshold value condition, and the second color-difference signal C calculated by a difference between the first color value and the second brightness signal Y;
synthesizing, with the processor, the first brightness signal Y and the second brightness signal Y to generate a third brightness signal Y and synthesizing the first color-difference signal C and the second color-difference signal C to generate a third color-difference signal C; and
performing, with the processor, a filtering process on the third brightness signal Y to enhance a high spatial frequency component and on the third color-difference signal C to enhance a low spatial frequency component.
2. The method according to claim 1 , further comprising:
performing a filtering process on the first brightness signal Y to enhance high spatial frequency components and on the first color-difference signal C to enhance low spatial frequency components.
3. The method according to claim 1 , the first color-difference signal C calculated by a difference between the third color value and the first brightness signal Y.
4. The method according to claim 1 , the first color being red, the second color being green, and the third color being blue.
5. The method according to claim 4 , each of the plurality of transformation formulas having R×K 1 +G×K 2 +B×K 1 , where R is the first color value, G is the second color value, B is the third color value, and K 1 +K 2 +K 1 =1.0.
6. The method according to claim 5 , the first transformation formula having a condition that one of K 1 and the K 2 corresponds to a largest value among the first color value, the second color value, and the third color value and that one of K 1 and K 2 has a smaller value than the other K 1 or the other K 2 in the first transformation formula.
7. The method according to claim 1 , the converting of the first color data further including converting the first color data into a second brightness signal Y and a second color-difference signal C, the second brightness signal Y being calculated by a second transformation formula selected from the plurality of transformation formulas based on the threshold value condition, and the second color-difference signal C calculated by a difference between the first color value and the second brightness signal Y, the method further comprising:
performing a filtering process on the first and second brightness signals Y to enhance high spatial frequency components and on the first and second color-difference signals C to enhance low spatial frequency components; and
synthesizing the first brightness signal Y and the second brightness signal Y, after both signals have been filtered to generate a third brightness signal Y and synthesizing the first color-difference signal C and the second color-difference signal C to generate a third color-difference signal C.
8. An image processing device processing an image that is output from a single-chip color image capturing sensor having a first color pixel, a second color pixel, and a third color pixel, the image processing device comprising:
a color interpolation processing unit, embodied on a non-transitory computer-readable medium, calculating first color data including a first color value, a second color value, and a third color value by an interpolation process, the first color data being color data of the first color pixel;
a YC converting unit, embodied on the non-transitory computer-readable medium, converting the first color data to a first brightness signal Y and a first color-difference signal C, the first brightness signal Y being calculated by a first transformation formula selected from a plurality of transformation formulas based on a threshold value condition determined by a difference between the first color value and a value proportionate to the second color value and by a difference between the third color value and the value proportionate to the second color value, and the first color-difference signal C calculated by a difference between the first color value and the first brightness signal Y, the YC converting unit further converting the first color data into a second brightness signal Y and a second color-difference signal C, the second brightness signal Y being calculated by a second transformation formula selected from the plurality of transformation formulas based on the threshold value condition, and the second color-difference signal C calculated by a difference between the first color value and the second brightness signal Y;
a synthesizing unit, embodied on the non-transitory computer-readable medium, synthesizing the first brightness signal Y and the second brightness signal Y to generate a third brightness signal Y and synthesizing the first color-difference signal C and the second color-difference signal C to generate a third color-difference signal C; and
a filtering unit, embodied on a non-transitory computer-readable medium, performing a filtering process on the third brightness signal Y to enhance a high spatial frequency component and on the third color-difference signal C to enhance a low spatial frequency component.
9. The image processing device according to claim 8 , further comprising:
a filtering unit performing a filtering process on the brightness signal Y to enhance high spatial frequency components and on the color-difference signal C to enhance low spatial frequency components.
10. The image processing device according to claim 8 , the YC converting unit further converting the first color data into a second brightness signal Y and a second color-difference signal C, the second brightness signal Y being calculated by a second transformation formula selected from the plurality of transformation formulas based on the threshold value condition, and the second color-difference signal C calculated by a difference between the first color value and the second brightness signal Y and, the image processing device further comprising:
a filtering unit performing a filtering process on the first and second brightness signals Y to enhance high spatial frequency components and on the first and second color-difference signals C to enhance low spatial frequency components; and
a synthesizing unit synthesizing the first brightness signal Y and the second brightness signal Y, after both signals have been filtered to generate a third brightness signal Y and synthesizing the first color-difference signal C and the second color-difference signal C to generate a third color-difference signal C.
11. An electronic apparatus comprising the image processing device according to claim 8 .
12. An image processing program stored in a non-transitory computer-readable storage medium, the image processing program allowing a computer to process an image that is output from a single-chip color image capturing sensor having a first color pixel, a second color pixel, and a third color pixel, the image processing program comprising:
a color interpolation portion calculating first color data including a first color value, a second color value, and a third color value by an interpolation process, the first color data being color data of the first color pixel;
a YC converting portion converting the first color data to a first brightness signal Y and a first color-difference signal C, the first brightness signal Y being calculated by a first transformation formula selected from a plurality of transformation formulas based on a threshold value condition determined by a difference between the first color value and a value proportionate to the second color value and by a difference between the third color value and the value proportionate to the second color value, and the first color-difference signal C calculated by a difference between the first color value and the first brightness signal Y, the YC converting portion further converting the first color data into a second brightness signal Y and a second color-difference signal C, the second brightness signal Y being calculated by a second transformation formula selected from the plurality of transformation formulas based on the threshold value condition, and the second color-difference signal C calculated by a difference between the first color value and the second brightness signal Y;
a synthesizing portion synthesizing the first brightness signal Y and the second brightness signal Y to generate a third brightness signal Y and synthesizing the first color-difference signal C and the second color-difference signal C to generate a third color-difference signal C; and
a filtering portion performing a filtering process on the third brightness signal Y to enhance a high spatial frequency component and on the third color-difference signal C to enhance a low spatial frequency component.
13. The image processing program according to claim 12 , further comprising:
a filtering portion performing a filtering process on the brightness signal Y to enhance high spatial frequency components and on the color-difference signal C to enhance low spatial frequency components.
14. The image processing program according to claim 12 , the YC converting portion further converting the first color data into a second brightness signal Y and a second color-difference signal C, the second brightness signal Y being calculated by a second transformation formula selected from the plurality of transformation formulas based on the threshold value condition, and the second color-difference signal C calculated by a difference between the first color value and the second brightness signal Y and, the image processing device further comprising:
a filtering portion performing a filtering process on the first and second brightness signals Y to enhance high spatial frequency components and on the first and second color-difference signals C to enhance low spatial frequency components; and
a synthesizing portion synthesizing the first brightness signal Y and the second brightness signal Y, after both signals have been filtered to generate a third brightness signal Y and synthesizing the first color-difference signal C and the second color-difference signal C to generate a third color-difference signal C.