Apparatus and method for enhancing image quality of image captured by using multiple color-filter aperture

1. An apparatus for enhancing image quality of an image, the apparatus comprising:

a region of interest (ROI) classifying unit classifying ROIs according to distances from an image capturing device with respect to the entire area of an input image captured by the image capturing device having an aperture with a plurality of openings employing different color filters by using a cluster-based image segmentation algorithm, the ROI classifying unit comprising:

a salient region selecting unit selecting a salient region corresponding to an object by applying a K-means clustering scheme and a hill-climbing scheme to the input image; and

an ROI labeling unit labeling blocks corresponding to the selected salient region among blocks segmented by using a block partitioning scheme, as ROIs;

a color channel registering unit determining motion vectors of color channels corresponding to the respective ROIs by analyzing a high frequency component of image data included in the ROIs, and moving the color channels based on the determined motion vectors to obtain a registered image of the respective ROIs;

an image fusing unit fusing the registered images of the respective ROIs to obtain a fused image; and

a smoothing unit smoothing the border between the ROIs existing in the fused image to enhance image quality of the fused image.

2. The apparatus of claim 1 , wherein the salient region selecting unit converts a color space of the input image into a CIELab color space, generates a 3D CIELab histogram, searches peaks with respect to every bin of the CIELab histogram by using the hill-climbing scheme, and classifies the salient region by using the K-means clustering scheme using K seeds based on the peaks, and the ROI labeling unit selects a reference block including clusters, and performs an operation of integrating adjacent blocks to a previous ROI when pixels are connected between the adjacent blocks including the clusters of the reference block and the reference block.

3. The apparatus of claim 1 , wherein the color channel registering unit sets one of the color channels as a reference channel, determines the motion vectors of the color channels with respect to the reference channel by using Equation A shown below, moves the color channels toward the reference channel according to the determined motion vectors and aligns them, moves the color channels toward the reference channel according to the determined motion vectors to align the color channels, and performs a center compensation operation of moving the color channels toward the center of gravity based on the reference channel and the position relationship of the aperture,

r d ( x,y )=δ( x−Δx,y−Δy )  Equation A

wherein r d is an inverse discrete Fourier transform (IDFT) of Rd as a cross-power spectrum, (Δx, Δy) are displacements of x and Y directions, respectively, F is a DCT of f indicating a certain color channel, and F, * is a complex conjugate of F,(iε{R,G,B}).

4. The apparatus of claim 1 , wherein the smoothing unit obtains a restored image and a segmented region from the fused image by using a soft-decision fusion and blending scheme, and then obtains an adaptively mixed image from the restored image and the segmented region through Equation B to Equation E shown below:

f

A

=

α

⁢

⁢

g

+

(

1

-

α

)

⁢

f

R

,

⁢

for

⁢

⁢

0

≤

α

≤

1

Equation

⁢

⁢

B

α

⁡

(

x

,

y

)

=

1

1

+

βσ

2

⁡

(

x

,

y

)

Equation

⁢

⁢

C

σ

2

⁡

(

x

,

y

)

=

1

PQ

⁢

∑

(

x

,

y

)

∈

S

⁢

(

f

⁡

(

x

,

y

)

-

m

xy

)

2

Equation

⁢

⁢

D

m

xy

=

1

PQ

⁢

∑

(

x

,

y

)

∈

S

⁢

f

⁡

(

x

,

y

)

Equation

⁢

⁢

E

wherein g is the restored image, f R is the segmented region, f A is the adaptively mixed image, α in Equation B is a weight factor calculated by Equation C, β in Equation C is a tuning parameter for making α greater than or equal to 0 or smaller than or equal to 1, σ 2 (x,y) is a local variance calculated by Equation D, P and Q are vertical and horizontal sizes of the regions, respectively, S is a quadrangular region including (x,y), and m xy is a local average of regions.

5. The apparatus of claim 1 , wherein the aperture comprises three openings employing red, blue, and green color filters, and the respective color channels are generated as incident light passes through the color filters.

6. A method for enhancing image quality of an image, the method comprising:

a region of interest (ROI) classifying operation of classifying ROIs according to distances from an image capturing device in an input image captured by the image capturing device having an aperture with a plurality of openings employing different color filters by using a cluster-based image segmentation algorithm, the ROI classifying operation comprising:

a salient region selecting operation of selecting a salient region corresponding to an object by applying a K-means clustering scheme and a hill-climbing scheme to the input image; and

an ROI labeling operation of labeling blocks corresponding to the selected salient region among blocks segmented by using a block partitioning scheme, as ROIs;

a color channel registering operation of determining motion vectors of color channels corresponding to the respective ROIs by analyzing a high frequency component of image data included in the ROIs, and moving the color channels based on the determined motion vectors to obtain a registered image of the respective ROIs;

an image fusing operation of fusing the registered images of the respective ROIs to obtain a fused image; and

a smoothing operation of smoothing the border between the ROIs existing in the fused image to enhance image quality of the fused image.

7. The method of claim 6 , wherein the salient region selecting operation comprises:

converting a color space of the input image into a CIELab color space and generating a 3D CIELab histogram;

searching peaks with respect to every bin of the CIELab histogram by using the hill-climbing scheme; and

classifying the salient region by using the K-means clustering scheme using K seeds based on the peaks; and

the ROI labeling operation comprises:

selecting a reference block comprising clusters; and

performing an operation of integrating adjacent blocks to a previous ROI when the adjacent blocks comprises the clusters of the reference block and pixels are connected between the adjacent blocks and the reference block.

8. The method of claim 6 , wherein the color channel registering operation comprises:

setting one of the color channels as a reference channel;

determining the motion vectors of the color channels with respect to the reference channel by using Equation A shown below;

moving the color channels toward the reference channel according to the determined motion vectors and aligning them; and

a center compensation operation of moving the color channels toward the center of gravity in consideration of the reference channel and the position relationship of the aperture,

r d ( x,y )=δ( x−Δx,y−Δy )  Equation A

wherein r d is an inverse discrete Fourier transform (IDFT) of Rd as a cross-power spectrum, (Δx, Δy) are displacements of x and Y directions, respectively, F is a DCT of f indicating a certain color channel, and F i* is a complex conjugate of F i (iε{R,G,B}).

9. The method of claim 6 , wherein the smoothing operation comprises:

obtaining a restored image and a segmented region from the fused image by using a soft-decision fusion and blending scheme; and

obtaining an adaptively mixed image from the restored image and the segmented region through Equation B to Equation E shown below:

f

A

=

α

⁢

⁢

g

+

(

1

-

α

)

⁢

f

R

,

⁢

for

⁢

⁢

0

≤

α

≤

1

Equation

⁢

⁢

B

α

⁡

(

x

,

y

)

=

1

1

+

βσ

2

⁡

(

x

,

y

)

Equation

⁢

⁢

C

σ

2

⁡

(

x

,

y

)

=

1

PQ

⁢

∑

(

x

,

y

)

∈

S

⁢

(

f

⁡

(

x

,

y

)

-

m

xy

)

2

Equation

⁢

⁢

D

m

xy

=

1

PQ

⁢

∑

(

x

,

y

)

∈

S

⁢

f

⁡

(

x

,

y

)

Equation

⁢

⁢

E

wherein g is the restored image, f R is the segmented region, f A is the adaptively mixed image, α in Equation B is a weight factor calculated by Equation C, β in Equation C is a tuning parameter for making a greater than or equal to 0 or smaller than or equal to 1, σ 2 (x,y) is a local variance calculated by Equation D, P and Q are vertical and horizontal sizes of the regions, S is a quadrangular region including (x,y), and m xy is a local average of regions.

10. The method of claim 6 , wherein the aperture comprises three openings employing red, blue, and green color filters, and the respective color channels are generated as incident light passes through the color filters.

11. A computer-readable medium containing program instructions for enhancing image quality of an image, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out the acts of:

a region of interest (ROI) classifying operation of classifying ROIs according to distances from an image capturing device in an input image captured by the image capturing device having an aperture with a plurality of openings employing different color filters by using a cluster-based image segmentation algorithm; the ROI classifying operation comprising:

a salient region selecting operation of selecting a salient region corresponding to an object by applying a K-means clustering scheme and a hill-climbing scheme to the input image; and

an ROI labeling operation of labeling blocks corresponding to the selected salient region among blocks segmented by using a block partitioning scheme, as ROIs;

a color channel registering operation of determining motion vectors of color channels corresponding to the respective ROIs by analyzing a high frequency component of image data included in the ROIs, and moving the color channels based on the determined motion vectors to obtain a registered image of the respective ROIs;

an image fusing operation of fusing the registered images of the respective ROIs to obtain a fused image; and

a smoothing operation of smoothing the border between the ROIs existing in the fused image to enhance image quality of the fused image.