Method and apparatus for processing image
View Patent ↗An image processing apparatus includes an adaptive interpolation device which converts a MFA pattern image into a quincuncial pattern image based on difference values, and interpolates color channels and an NIR channel, based on difference values of the converted quincuncial pattern image in vertical and horizontal pixel directions; a frequency compensation device which obtains a high-resolution MFA image using high-frequency and medium-frequency components of a high-resolution base image, based on linear regression analysis and compared energy levels of MFA channel images to an energy level of a base image; and a channel interference suppression device which removes color distortion generated between each channel of the high-resolution MFA image, and another channel of the high-resolution MFA image and a base channel using a weighted average of pixel value differences between each channel of the high-resolution MFA image, and the other channel of the high-resolution MFA image and the base channel.
1. An image processing apparatus comprising:
an adaptive interpolation device configured to convert a multispectral filter array (MFA) pattern image into a quincuncial pattern image based on difference values in a diagonal pixel direction, and interpolate color channels and a near infrared (NIR) channel of the MFA pattern image to a maximum resolution, based on difference values of the converted quincuncial pattern image in vertical pixel directions and horizontal pixel directions;
a frequency compensation device configured to obtain a high-resolution MFA image using high-frequency components and medium-frequency components of a high-resolution base image corresponding to high-frequency components and medium-frequency components of a low-resolution MFA image interpolated by the adaptive interpolation device, based on linear regression analysis and a result of comparing energy levels of MFA channel images to an energy level of a base image; and
a channel interference suppression device configured to remove color distortion generated between each channel of the high-resolution MFA image, and a first channel of the high-resolution MFA image and a base channel using a weighted average of pixel value differences between each channel of the high-resolution MFA image, and a second channel of the high-resolution MFA image and the base channel.
2. The image processing apparatus of claim 1 , wherein the quincuncial pattern image is an image having a same geometric model as a Bayer pattern image, and having different channel values from the Bayer pattern image.
3. The image processing apparatus of claim 1 , wherein, when the MFA pattern image is converted into the quincuncial pattern image, the adaptive interpolation device uses a sum of a first weighted average of values of pixels spaced apart from a certain pixel (m,n) of the MFA pattern image by one pixel in a diagonal direction, and a second weighted average, obtained by calculating a weighted average of difference values between the certain pixel (m,n) of the MFA pattern image and pixels spaced apart from the certain pixel (m,n) of the MFA pattern image by two pixels in the diagonal direction.
4. The image processing apparatus of claim 3 , wherein each weight used to calculate the first weighted average and the second weighted average is inversely proportional to a difference value in each pixel direction.
5. The image processing apparatus of claim 1 , wherein the frequency compensation device generates the high-resolution MFA image by comparing the medium-frequency components and the high-frequency components of the low-resolution MFA image interpolated by the adaptive interpolation device to medium-frequency components and high-frequency components of the base image in each of vertical, horizontal, and diagonal directions, calculating a weighted sum of corresponding parts, and reconstructing the medium-frequency components and the high-frequency components of the low-resolution MFA image interpolated by the adaptive interpolation device.
6. The image processing apparatus of claim 1 , wherein the frequency compensation device provides a weight to each of the medium-frequency components and the high-frequency components of the low-resolution MFA image and medium-frequency components and high-frequency components of the base image, and
wherein the weight is set such that a large weight is provided to an edge of the low-resolution MFA image or an edge of the base image, whichever has a higher energy level.
7. The image processing apparatus of claim 1 , wherein the frequency compensation device calculates a ratio between energy levels of medium-frequency components and high-frequency components of the base image and the MFA image, using the linear regression analysis, and provides a weight for correcting an energy level difference.
8. The image processing apparatus of claim 7 , wherein the frequency compensation device removes color distortion between channels by strongly reconstructing medium-frequency components and high-frequency components of a channel having a high energy level of the high-frequency components, and weakly reconstructing medium-frequency components and high-frequency components of a channel having a low energy level of the high-frequency components using the weight for correcting the energy level difference, so as to allow edges of channels to be uniform in consideration of energy levels of red (R), green (G), blue (B), and NIR channels of the MFA image.
9. An image processing method comprising:
converting a multispectral filter array (MFA) pattern image into a quincuncial pattern image based on difference values in a diagonal pixel direction, and interpolating color channels and a near infrared (NIR) channel of the MFA pattern image to a maximum resolution, based on difference values of the converted quincuncial pattern image in vertical pixel directions and horizontal pixel directions, using an adaptive interpolation device;
obtaining a high-resolution MFA image using high-frequency components and medium-frequency components of a high-resolution base image corresponding to high-frequency and medium-frequency components of a low-resolution MFA image interpolated by the adaptive interpolation device, based on linear regression analysis and a result of comparing energy levels of MFA channel images to an energy level of a base image, using a frequency compensation device; and
removing color distortion generated between each channel of the high-resolution MFA image, and a first channel of the high-resolution MFA image and a base channel using a weighted average of pixel value differences between each channel of the high-resolution MFA image, and a second channel of the high-resolution MFA image and the base channel, using a channel interference suppression device.
10. The image processing method of claim 9 , wherein the quincuncial pattern image is an image having a same geometric model as a Bayer pattern image, and having different channel values from the Bayer pattern image.
11. The image processing method of claim 9 , wherein, when the MFA pattern image is converted into the quincuncial pattern image, a sum of a first weighted average of values of pixels spaced apart from a certain pixel (m,n) of the MFA pattern image by one pixel in a diagonal direction, and a second weighted average, obtained by calculating a weighted average of difference values between the certain pixel (m,n) of the MFA pattern image and pixels spaced apart from the certain pixel (m,n) of the MFA pattern image by two pixels in the diagonal direction, is used.
12. The image processing method of claim 11 , wherein each weight used to calculate the first weighted average and the second weighted average is inversely proportional to a difference value in each pixel direction.
13. The image processing method of claim 9 , wherein the obtaining of the high-resolution MFA image comprises generating the high-resolution MFA image by comparing the medium-frequency components and the high-frequency components of the low-resolution MFA image interpolated by the adaptive interpolation device to medium-frequency components and high-frequency components of the base image in each of vertical, horizontal, and diagonal directions, calculating a weighted sum of corresponding parts, and reconstructing the medium-frequency components and the high-frequency components of the low-resolution MFA image interpolated by the adaptive interpolation device.
14. The image processing method of claim 9 , wherein the obtaining of the high-resolution MFA image comprises providing a weight to each of the medium-frequency components and the high-frequency components of the low-resolution MFA image and medium-frequency components and high-frequency components of the base image, and
wherein the weight is set such that a large weight is provided to an edge of the low-resolution MFA image or an edge of the base image, whichever has a higher energy level.
15. The image processing method of claim 9 , wherein the obtaining of the high-resolution MFA image comprises calculating a ratio between energy levels of medium-frequency components and high-frequency components of the base image and the MFA image, using the linear regression analysis, and providing a weight for correcting an energy level difference.
16. The image processing method of claim 15 , wherein the obtaining of the high-resolution MFA image comprises removing color distortion between channels by strongly reconstructing medium-frequency components and high-frequency components of a channel having a high energy level of the high-frequency components, and weakly reconstructing medium-frequency components and high-frequency components of a channel having a low energy level of the high-frequency components using the weight for correcting the energy level difference, so as to allow edges of channels to be uniform in consideration of energy levels of red (R), green (G), blue (B), and NIR channels of the MFA image.