Multi-image sharpening and denoising using lucky imaging
The multi-image sharpening and denoising technique described herein creates a clean (low-noise, high contrast), detailed image of a scene from a temporal series of images of the scene. The technique employs a process of image alignment to remove global and local camera motion plus a novel weighted image averaging procedure that avoids sacrificing sharpness to create a resultant high-detail, low-noise image from the temporal series. In addition, the multi-image sharpening and denoising technique can employ a dehazing procedure that uses a spatially varying airlight model to dehaze an input image.
1. A computer-implemented process for creating an improved image of objects in the distance from multiple images of the same scene, comprising:
using a computing device for:
inputting multiple images of a scene,
performing a global translational alignment of each input image to a single image of the input multiple images of the scene;
averaging the pixels of each of the resulting globally aligned images to create a globally aligned average image;
computing a local alignment of each resulting globally aligned image by computing a pixel-wise optical flow to the globally aligned average image initialized by the resulting globally aligned image for each input image;
for each pixel location, for each globally and locally aligned image, determining a normalized pixel-wise weight for each corresponding pixel in each image; and
creating a weighted average image from the set of normalized pixel-wise weights and the resulting globally and locally aligned images.
2. The computer-implemented process of claim 1 wherein the normalized pixel-wise weight is a function of local sharpness in each globally and locally aligned image.
3. The computer-implemented process of claim 2 , wherein more pixel samples are used in areas of high local texture than in areas of low local texture when computing the normalized pixel-wise weight.
4. The computer-implemented process of claim 1 , wherein the normalized pixel-wise weight is a function of how much resampling the pixels have undergone in each globally aligned and locally aligned image.
5. The computer-implemented process of claim 4 , wherein fractional pixel alignments are assigned less weight in computing the normalized pixel-wise weight.
6. The computer-implemented process of claim 4 , wherein fractional pixel alignments are assigned less weight relative to a color sampling grid.
7. The computer-implemented process of claim 1 , wherein the normalized pixel-wise weight is a function of sensor dust.
8. The computer-implemented process of claim 7 , wherein a mask is used to remove sensor dust from the weighted average image.
9. The computer-implemented process of claim 1 , further comprising post-processing the weighted average image by dehazing the weighted average image.
10. The computer-implemented process of claim 9 , wherein dehazing the weighted average image, further comprises employing a spatially varying airlight color when performing dehazing.
11. The computer-implemented process of claim 10 , wherein the spatially varying air light color varies by image scan line.
12. The computer-implemented process of claim 10 , further comprising post-processing the weighted average image by contrast expanding the weighted average image.
13. A computer-implemented process for creating an improved image from a multiple images of the same scene, comprising:
using a computing device for:
inputting multiple images of a scene,
performing a global translational alignment of each image to a single image of the input multiple images;
averaging the pixels of each the resulting globally aligned images to create a globally aligned average image;
for each pixel location, for each global alignment result of each image, determining a normalized pixel-wise weight for each corresponding pixel in each image; and
creating a weighted average image from the set of normalized pixel-wise weights and the resulting globally aligned images.
14. The computer-implemented process of claim 13 , wherein the normalized pixel-wise weight is a function of selecting pixels based on local sharpness.
15. The computer-implemented process of claim 14 , further comprising averaging pixels in areas of high texture less than averaging pixels in low texture areas.
16. The computer-implemented process of claim 13 , wherein the normalized pixel-wise weight is a function of the resampling of the pixels in each globally aligned image by weighting fractional pixel shifts less than non-fractional pixel shifts.
17. The computer-implemented process of claim 13 , further comprising dehazing the weighted average image by employing a spatially varying air light color when performing dehazing.
18. A system for dehazing an image using an air light model, comprising:
a general purpose computing device;
a computer program comprising program modules executable by the general purpose computing device, wherein the computing device is directed by the program modules of the computer program to,
input an image;
compute an air light color by using a spatially varying air light model that varies air light by horizontal image scanline and that estimates a dark channel value as the darkest value per horizontal image scan line that is used to derive the presence and magnitude of haze, wherein the dark channel value is filtered in a vertical direction; and
dehaze the input image by subtracting the air light that varies by horizontal scanline from the corresponding horizontal scanlines of input image.
19. The system of claim 18 , further comprising a module for contrast expanding the dehazed image.