IP Library Granted Patent US 9,135,679
Granted Patent B2
US 9,135,679 · App. 13/948,256 · Granted Sep 15, 2015

Apparatus and method for color restoration

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,135,679
App. No.
13/948,256
Granted
Sep 15, 2015
Kind
B2
Abstract

Provided is a method and apparatus for restoring color of an input image. The method includes: separating multiple band information of a near infrared (NIR) channel from information of the input image by using a correlation between a color channel and the NIR channel in the input image; obtaining estimated invisible light band information of the NIR channel from the multiple band information of the NIR channel and estimated multiple band information of the NIR channel which is estimated from the color channel; estimating invisible light band information of the color channel based on the estimated invisible light band information of the NIR channel; and restoring a visible light band image of the color channel by removing the estimated invisible light band information of the color channel from multiple band information of the color channel included in the information of the input image.

Claims (114)

1. A color restoration method comprising:

separating multiple band information of a near infrared (NIR) channel from information of an input image by using variables indicating a correlation between a color channel and the NIR channel in the input image;

obtaining estimated invisible light band information of the NIR channel from the multiple band information of the NIR channel and estimated multiple band information of the NIR channel which is estimated from the color channel;

estimating invisible light band information of the color channel based on the estimated invisible light band information of the NIR channel; and

restoring a visible light band image of the color channel by removing the estimated invisible light band information of the color channel from multiple band information of the color channel included in the information of the input image.

2. The color restoration method of claim 1 , further comprising calculating a difference between visible light band information and estimated visible light band information of an input image,

wherein the separating the multiple band information of the NIR channel from the input image is performed based on a case in which the difference is the smallest.

3. The color restoration method of claim 2 , further comprising

obtaining adaptive chromaticity coefficients for the case in which the difference is the smallest; and

suppressing interference between the color channel and the NIR channel using the adaptive chromaticity coefficients.

4. The color restoration method of claim 2 , wherein the obtaining the estimated invisible light band information of the NIR channel is performed by determining and removing noise of the multiple band information of the NIR channel.

5. The color restoration method of claim 4 , wherein the noise is determined by obtaining a difference between the multiple band information of the NIR channel and a value obtained by multiplying the estimated multiple band information of the NIR channel by a value which is greater than 0 and smaller than 1.

6. The color restoration method of claim 1 , wherein the multiple band information of the color channel comprises visible light band information and invisible light band information of the color channel, and

wherein the multiple band information of the NIR channel comprises visible light information and invisible light information of the NIR channel.

7. The color restoration method of claim 1 , wherein the estimated invisible light band information of the NIR channel is obtained by using an equation λK(N all −r{circumflex over (N)} all )={circumflex over (N)} nir ,

wherein

λ

=

1

r

×

N

^

nir

(

1

r

-

1

)

×

N

all

×

K

+

N

^

nir

,

and

wherein N all is the multiple band information of the NIR channel, N^ all is the estimated multiple band information estimated from the color channel, and r is a constant greater than 0 and smaller than 1, and K is another constant.

8. The color restoration method of claim 1 , further comprising suppressing color restoration in a region that exceeds a threshold value by applying a following weight curve to the NIR channel to perform light saturation compensation,

curve

=

e

4

×

(

x

-

threshold

)

2

normal

2

,

wherein x is a pixel value of an NIR image, normal is a constant that is a normalized term, threshold =MAX(N i,j input )*0.7 and N i,j input is a pixel value of the NIR image.

9. The color restoration method of claim 1 , further comprising performing light saturation compensation on the NIR channel in the restored visible light band image.

10. A color restoration apparatus comprising:

a channel correlation checking unit configured to separate multiple band information of a near infrared (NIR) channel from information of an input image by using variables indicating a correlation between a color channel and the NIR channel in the input image;

an adaptive gain control (AGC) unit configured to obtain estimated invisible light band information of the NIR channel from the multiple band information of the NIR channel and estimated multiple band information of the NIR channel which is estimated from the color channel; and

a visible light band information restoring unit configured to estimate invisible light band information of the color channel based on the estimated invisible light band information of the NIR channel, and restore a visible light band image of the color channel by removing the estimated invisible light band information of the color channel from multiple band information of the color channel included in the information of the input image.

11. The color restoration apparatus of claim 10 , further comprising an adaptive chromaticity coefficient generating unit configured to calculate a difference between visible light band information and estimated visible light band information of an input image,

wherein the channel correlation checking unit is configured to separate the multiple band information of the NIR channel from the input image based on a case in which the difference is the smallest.

12. The color restoration apparatus of claim 11 , wherein the adaptive chromaticity coefficient generating unit is configured to obtain adaptive chromaticity coefficients for the case in which the difference is the smallest, and suppress interference between the color channel and the NIR channel using the adaptive chromaticity coefficients.

13. The color restoration apparatus of claim 11 , wherein the AGC unit is configured to obtain the estimated invisible light band information of the NIR channel by determining and removing noise of the multiple band information of the NIR channel.

14. The color restoration apparatus of claim 13 , wherein the AGC unit is configured to determine the noise by obtaining a difference between the multiple band information of the NIR channel and a value obtained by multiplying the estimated multiple band information of the NIR channel by a value which is greater than 0 and smaller than 1.

15. The color restoration apparatus of claim 10 , wherein the multiple band information of the color channel comprises visible light band information and invisible light band information of the color channel, and

wherein the multiple band information of the NIR channel comprises visible light information and invisible light information of the NIR channel.

16. The color restoration apparatus of claim 10 , wherein the AGC unit is configured to obtain the invisible light band information of the NIR channel by using an equation λK(N all −r{circumflex over (N)} all )={circumflex over (N)} nir ,

wherein

λ

=

1

r

×

N

^

nir

(

1

r

-

1

)

×

N

all

×

K

+

N

^

nir

,

and

wherein N all is the multiple band information of the NIR channel, N^ all is the estimated multiple band information estimated from the color channel, and r is a constant greater than 0 and smaller than 1, and K is another constant.

17. The color restoration apparatus of claim 10 , further comprising a light saturation controlling unit configured to suppress color restoration in a region that exceeds a threshold value by applying a following weight curve to the NIR channel to perform light saturation compensation,

curve

=

e

4

×

(

x

-

threshold

)

2

normal

2

wherein x is a pixel value of an NIR image, normal is a constant that is a normalized term, threshold =MAX(N i,j input )*0.7 and N i,j input a pixel value of the NIR image.

18. The color restoration apparatus of claim 10 , further comprising a light saturation controlling unit configured to perform light saturation compensation on the NIR channel in the restored visible light band image.

Assignments (6)
CHANGE OF NAME Recorded Aug 10, 2023
From: HANWHA TECHWIN CO., LTD.
To: HANWHA VISION CO., LTD.
Reel/Frame 064549/0075 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 26, 2019
From: HANWHA AEROSPACE CO., LTD.
To: HANWHA TECHWIN CO., LTD.
Reel/Frame 049013/0723 →
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICATION NUMBER 10/853,669. IN ADDITION PLEASE SEE EXHIBIT A PREVIOUSLY RECORDED ON REEL 046927 FRAME 0019. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Jan 17, 2019
From: HANWHA TECHWIN CO., LTD.
To: HANWHA AEROSPACE CO., LTD.
Reel/Frame 048496/0596 →
CHANGE OF NAME Recorded Aug 24, 2018
From: HANWHA TECHWIN CO., LTD
To: HANWHA AEROSPACE CO., LTD.
Reel/Frame 046927/0019 →
CHANGE OF NAME Recorded Jul 24, 2015
From: SAMSUNG TECHWIN CO., LTD.
To: HANWHA TECHWIN CO., LTD.
Reel/Frame 036254/0911 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2013
From: CHOI, EUN-CHEOL; KANG, MOON-GI; OH, HYUN-MOOK; PARK, CHUL-HEE
To: SAMSUNG TECHWIN CO., LTD.; INDUSTRY-ACADEMIC COOPERATION FOUNDATION, YONSEI UNIVERSITY
Reel/Frame 030853/0454 →