Granted Patent B2

US 9,330,446 · App. 13/953,971 · Granted May 3, 2016

Abstract

Claims (65)

1. A method of processing an image, the method comprising:

generating an overlap image by overlapping a first image having a predetermined exposure time and a second image having an exposure time different from that of the first image;

detecting a region of interest (ROI) in which a motion blur occurs in the overlap image;

performing a motion blur removing operation that changes an image in the ROI to any one of the first image and the second image by applying a first machine learning algorithm; and

performing an output image processing operation that outputs a wide dynamic range (WDR) image by changing a part, excluding an object in the image in the ROI, of an image from which motion blur has been removed, to a background image of a previous WDR image by applying a second machine learning algorithm.

2. The method of claim 1 , wherein the detecting of the ROI comprises:

detecting an edge in the overlap image; and

detecting the ROI including an outline of the detected edge.

3. The method of claim 1 , wherein the motion blur removing operation comprises:

determining whether the object is a moving object, through an ROI image comparison between previous and subsequent frames;

calculating a mean brightness level of the image in the ROI when a moving object does not exist in the image in the ROI;

changing the image in the ROI to the first image when the mean brightness level is equal to or less than a first threshold; and

changing the image in the ROI to the second image when the mean brightness level is greater than the first threshold.

4. The method of claim 1 , wherein the motion blur removing operation comprises:

determining whether the object is a moving object;

determining whether the moving object has a learned predetermined pattern when the moving object exists in the image in the ROI;

changing the image in the ROI to the first image when it is determined that the moving object has the learned predetermined pattern and when the moving object is a human being; and

changing the image in the ROI to the second image when it is determined that the moving object has the learned predetermined pattern and when the moving object is not a human being.

5. The method of claim 1 , wherein the motion blur removing operation comprises:

determining whether the object is a moving object;

determining whether the moving object has a learned predetermined pattern when the moving object exists in the image in the ROI;

determining a speed of the moving object when it is determined that the moving object does not have the learned predetermined pattern;

changing the image in the ROI to the first image when the speed of the moving object is equal to or less than a second threshold; and

changing the image in the ROI to the second image when the speed of the moving object is greater than the second threshold.

6. The method of claim 1 , wherein the output image processing operation comprises:

calculating a maximum brightness level and a minimum brightness level of the image in the ROI of the image from which motion blur has been removed;

differently adjusting a brightness ratio of the previous WDR image to the background image according to the maximum brightness level and the minimum brightness level; and

outputting the WDR image by changing the part, excluding the object in the image in the ROI, to the background image for which the brightness ratio has been differently adjusted.

7. The method of claim 6 , wherein the different adjusting of the brightness ratio comprises:

decreasing the brightness ratio of the background image as the part, excluding the object in the image in the ROI, approaches the maximum brightness level; and

increasing the brightness ratio of the background image as the part, excluding the object in the image in the ROI, approaches the minimum brightness level.

8. An apparatus for processing an image, the apparatus comprising:

at least one processor which implements:

an overlapping unit configured to generate an overlap image by overlapping a first image having a predetermined exposure time and a second image having an exposure time different from that of the first image;

a detection unit configured to detect a region of interest (ROI) in which a motion blur occurs in the overlap image;

a motion blur removing unit configured to change an image in the ROI to any one of the first image and the second image by applying a first machine learning algorithm; and

an output image processing unit configured to output a WDR image by changing a part, excluding an object in the image in the ROI, of an image from which motion blur has been removed, to a background image of a previous WDR image by applying a second machine learning algorithm.

9. The apparatus of claim 8 , wherein the detection unit further comprises an edge detection unit configured to detect an edge in the overlap image, and

the ROI including an outline of the detected edge is detected.

10. The apparatus of claim 8 , wherein the motion blur removing unit comprises:

an object determination unit configured to determine whether the object is a moving object, through an ROI image comparison between previous and subsequent frames;

a first brightness level calculation unit configured to calculate a mean brightness level of the image in the ROI when a moving object does not exist in the image in the ROI;

a selection unit configured to select the first image or the second image according to the mean brightness level; and

a first changing unit configured to change the image in the ROI to the first image when the mean brightness level is equal to or less than a first threshold, and change the image in the ROI to the second image when the mean brightness level is greater than the first threshold.

11. The apparatus of claim 8 , wherein the motion blur removing unit comprises:

an object determination unit configured to determine whether the object is a moving object;

a pattern recognition unit configured to determine whether the moving object is a human being having a learned predetermined pattern when the moving object exists in the image in the ROI;

a selection unit configured to select the first image or the second image according to a result of the pattern recognition; and

a second changing unit configured to change the image in the ROI to the first image when it is determined that the moving object is a human being, and change the image in the ROI to the second image when it is determined that the moving object is not a human being.

12. The apparatus of claim 8 , wherein the motion blur removing unit comprises:

an object determination unit configured to determine whether the object is a moving object;

a pattern recognition unit configured to determine whether the moving object has a learned predetermined pattern when the moving object exists in the image in the ROI;

a speed determination unit configured to determine a speed of the moving object when it is determined that the moving object does not have the learned predetermined pattern;

a selection unit configured to select the first image or the second image according to the speed determination result; and

a third changing unit configured to change the image in the ROI to the first image when the speed of the moving object is equal to or less than a second threshold, and change the image in the ROI to the second image when the speed of the moving object is greater than the second threshold.

13. The apparatus of claim 8 , wherein the output image processing unit comprises:

a second brightness level calculation unit configured to calculate a maximum brightness level and a minimum brightness level of the image in the ROI of the image from which motion blur has been removed;

an adjustment unit configured to differently adjust a brightness ratio of the previous WDR image to the background image according to the maximum brightness level and the minimum brightness level; and

a fourth changing unit configured to output the WDR image by changing the part, excluding the object in the image in the ROI, to the background image for which the brightness ratio has been differently adjusted.

14. The apparatus of claim 13 , wherein the adjustment unit is further configured to decrease the brightness ratio of the background image as the part, excluding the object in the image in the ROI, approaches the maximum brightness level, and configured to increase the brightness ratio of the background image as the part, excluding the object in the image in the ROI, approaches the minimum brightness level.

15. A non-transitory computer-readable storage medium having stored therein program instructions, which when executed by a computer, perform a method of processing an image, the method comprising:

generating an overlap image by overlapping a first image having a predetermined exposure time and a second image having an exposure time different from that of the first image;

detecting a region of interest (ROI) in which a motion blur occurs in the overlap image;

a motion blur removing operation of changing an image in the ROI to any one of the first image and the second image by applying a first machine learning algorithm; and

performing an output image processing operation that outputs a wide dynamic range (WDR) image by changing a part, excluding an object in the image in the ROI, of an image from which motion blur has been removed, to a background image of a previous WDR image by applying a second machine learning algorithm.

Assignments (6)