Tone mapping for low-light video frame enhancement
View Patent ↗A technique is provided for generating sharp, well-exposed, color images front low-light images. A series of under-exposed images is acquired. A mean image is computed and a sum image is generated each based on the series of under-exposed images. Chrominance variables of pixels of the mean image are mapped to chrominance variables of pixels of the sum image. Chrominance values of pixels within the series of under-exposed images are replaced with chrominance values of the sum image. A set of sharp, well-exposed, color images is generated based on the series of under-exposed images with replaced chrominance values.
1. An image processing method, comprising:
acquiring a first well-exposed image frame of a scene;
acquiring additional image frames of the same scene;
wherein the additional image frames include a second image frame and a third image frame;
generating one or more combined images based on the well-exposed image frame of the scene and at least one of the additional image frames;
determining a frame-to-frame motion between the second image frame and the third image frame; and
responsive to said frame-to-frame motion exceeding a threshold, acquiring a second well-exposed image frame of the scene.
2. The image processing method of claim 1 , further comprising:
after acquiring the second well-exposed image frame, acquiring second additional image frames of the scene;
generating one or more combined images based on the second well-exposed image frame and at least one of the second additional image frames; and
acquiring a third well-exposed image frame responsive to the number of second additional image frames exceeding a threshold.
3. The image processing method of claim 1 ,
wherein the frame-to-frame motion is determined from a registration of the additional image frames.
4. The image processing method of claim 1 ,
wherein the additional image frames are frames that are exposed using an exposure determined from a maximum frame-rate.
5. The image processing method of claim 1 ,
wherein the one or more combined images are generated based on the well-exposed image frame of the scene and the additional image frames of the same scene until one or more conditions are satisfied, wherein the one or more conditions include:
the frame-to-frame motion between two particular additional image frames, from the additional image frames, exceeds the threshold, or
a certain number of frames, from the additional image frames, have been acquired.
6. The image processing method of claim 1 ,
wherein generating one or more combined images includes generating a first combined image by:
extracting brightness and/or color information from the well-exposed image frame; and
using the brightness and/or color information from the well-exposed image frame to enhance the second image frame.
7. The image processing method of claim 1 ,
wherein generating one or more combined images includes generating a second combined image by:
extracting brightness and/or color information from the first combined image; and
using the brightness and/or color information from the first combined image to enhance the third image frame.
8. The image processing method of claim 1 ,
wherein an exposure time for capturing the additional image frames is determined using an implementation of an auto-exposure algorithm executed when a camera is set in a video mode.
9. The image processing method of claim 8 ,
wherein the camera is configured to acquire the first well-exposed image frame using an auto-exposure set for a time period that is larger than one imposed by a maximum frame rate.
10. An image acquisition and processing device, comprising:
an image acquisition unit configured to:
acquiring a first well-exposed image frame of a scene;
acquiring additional image frames of the same scene;
wherein the additional image frames include a second image frame and a third image frame;
an image combining unit configured to:
generating one or more combined images based on the well-exposed image frame of the scene and at least one of the additional image frames;
determining a frame-to-frame motion between the second image frame and the third image frame; and
responsive to said frame-to-frame motion exceeding a threshold, acquiring a second well-exposed image frame of the scene.
11. The image acquisition and processing device of claim 10 , wherein the image combining unit is further configured to:
after acquiring the second well-exposed image frame, acquiring second additional image frames of the scene;
generating one or more combined images based on the second well-exposed image frame and at least one of the second additional image frames; and
acquiring a third well-exposed image frame responsive to the number of second additional image frames exceeding a threshold.
12. The image acquisition and processing device of claim 10 ,
wherein the frame-to-frame motion is determined from a registration of the additional image frames.
13. The image acquisition and processing device of claim 10 ,
wherein the additional image frames are frames that are exposed using an exposure determined from a maximum frame-rate.
14. The image acquisition and processing device of claim 10 ,
wherein the one or more combined images are generated based on the well-exposed image frame of the scene and the additional image frames of the same scene until one or more conditions are satisfied, wherein the one or more conditions include:
the frame-to-frame motion between two particular additional image frames, from the additional image frames, exceeds the threshold, or
a certain number of frames, from the additional image frames, has been recorded.
15. The image acquisition and processing device of claim 10 ,
wherein the one or more combined images are generated by extracting brightness and/or color information from the well-exposed image frame to enhance the at least one of the additional image frames.
16. The image acquisition and processing device of claim 10 ,
wherein an exposure time for capturing the additional image frames is determined using an implementation of an auto-exposure algorithm executed when a camera is set in a video mode.
17. The image acquisition and processing device of claim 16 ,
wherein the camera is configured to acquire the first well-exposed image frame using an auto-exposure set for a time period that is larger than one imposed by a maximum frame rate.
18. A non-transitory computer readable storage medium storing one or more instructions which, when executed by a processor, cause the processor to perform:
acquiring a first well-exposed image frame of a scene;
acquiring additional image frames of the same scene;
wherein the additional image frames include a second image frame and a third image frame;
generating one or more combined images based on the well-exposed image frame of the scene and at least one of the additional image frames;
determining a frame-to-frame motion between the second image frame and the third image frame; and
responsive to said frame-to-frame motion exceeding a threshold, acquiring a second well-exposed image frame of the scene.
19. The non-transitory computer readable storage medium of claim 18 ,
wherein the instructions include instructions for:
after acquiring the second well-exposed image frame, acquiring second additional image frames of the scene;
generating one or more combined images based on the second well-exposed image frame and at least one of the second additional image frames; and
acquiring a third well-exposed image frame responsive to the number of second additional image frames exceeding a threshold.
20. The non-transitory computer readable storage medium of claim 18 ,
wherein the frame-to-frame motion is determined from a registration of the additional image frames.
21. The non-transitory computer readable storage medium of claim 18 ,
wherein the additional image frames are frames that are exposed using an exposure determined from a maximum frame-rate.
22. The non-transitory computer readable storage medium of claim 18 ,
wherein the one or more combined images are generated based on the well-exposed image frame of the scene and the additional image frames of the same scene until one or more conditions are satisfied:
the frame-to-frame motion between two particular additional image frames, from the additional image frames, exceeds the threshold, or
a certain number of frames, from the additional image frames, has been recorded.
23. The non-transitory computer readable storage medium of claim 18 ,
wherein the one or more combined images are generated by extracting brightness and/or color information from the well-exposed image frame to enhance the at least one of the additional image frames.
24. The non-transitory computer readable storage medium of claim 18 ,
wherein an exposure time for capturing the additional image frames is determined using an implementation of an auto-exposure algorithm executed when a camera is set in a video mode.
25. The non-transitory computer readable storage medium of claim 24 ,
wherein the camera is configured to acquire the first well-exposed image frame using an auto-exposure set for a time period that is larger than one imposed by a maximum frame rate.