Automated video data fusion method
A method for mitigating image distortions induced by optical wave propagation through a random media (e.g., atmospheric turbulence or volume of water) from a stream of video data provided by a single shortexposure image sensor is described. The method is based on the two following sequential steps: (1) enhancement of the raw video stream and (2) fusion of the enhanced stream using the lucky region fusion (LRF) technique. The first step enhances features of the raw image stream the LRF method success is based on and especially mitigates the effect of low light level, aerosol pollution, dust, haze, and other deteriorating factors. The second step, fusion of the enhanced stream, is realized by sequentially merging image regions with highest quality within a temporal buffer into a single image before sliding the temporal window forward. The process is continuously repeated in order to generate a stream of fused images. The resulting fused stream hence has an image quality superior to that of any image within the buffer and demonstrates improved contrast as well as increased detail visualization. In addition, the disclosed invention offers a method for automated extraction of random media (atmospheric turbulence for example) characteristics needed for optimizing the LRF method performance. Based solely on analysis of the enhanced video stream, this has the advantage to eliminate the need for turbulence strength characterization devices (e.g., scintillometer) and it allows the invention to provide an optimal fused stream even when operating within an evolving environment.
1. A method for mitigating image distortions which comprises:
enhancing a raw video stream, comprising
static noise reduction;
shaking/jitter compensation;
dynamic noise reduction; and
contrast enhancement;
fusing said enhanced video stream using a lucky region fusion technique, comprising
computing a fusion kernel size;
determining quality metrics for regions of images in the enhanced video stream;
determining quality metrics for regions of images of a previously fused video stream;
computing anisotropic gain which performs a comparison of the image quality metrics for regions of images in the enhanced video stream, and regions of images in the previously fused video stream;
selecting best image quality regions among the enhanced and the previously fused video stream;
fusing the selected image regions into a new fused video stream;
determining global image quality metrics of the new fused video stream;
comparing the global quality metrics of the images in the new and the previously fused video streams; and
updating an image in the new fused video stream only when the quality metrics are superior by a predetermined threshold;
iteratively repeating the steps of enhancing and fusing, whereby a stream of fused images is generated;
extracting random media characteristics from said stream of fused images automatically.
2. A method for mitigating image distortions as recited in claim 1 wherein the fusing of the enhanced video stream step is performed by sequentially merging image regions with highest quality within a temporal buffer into a single image.
3. A method for mitigating image distortions as recited in claim 1 wherein the step of generating a stream of fused images is performed by moving the highest quality image from the temporal buffer after it has been fused and then sliding a temporal window forward.
4. A method for mitigating image distortions as recited in claim 1 wherein the step of automated random media extraction is performed by analyzing the stream of fused images and automatically removing random media characteristics including but not limited to atmospheric turbulence.