Method for producing super-resolution images and nonlinear digital filter for implementing same
View Patent ↗A method and a digital filter, for use with photo and video images, includes using a camera or video camera equipped with sensors and an electronic shutter to capture a plurality of frames of low resolution and producing one frame of high resolution. A plurality of frames are exposed. Initial images are in the form of a continuous sequence of frames with high-speed capture. The frequency of the frames is inversely proportional to the magnitude of that part of the light-sensitive region of the sensor that is being scanned. The initial images are aligned and an enhanced image is produced. The enhanced image is filtered using a nonlinear filter which includes a neural network that is pretrained using a test image including radial and sinusoidal test charts, as well as reference points.
1. A method of image super-resolution comprising:
exposing, at a predetermined frame rate, a digital imaging sensor to generate a plurality of low resolution frames;
aligning the plurality of low resolution frames;
forming an enlarged image by combining the low resolution frames; and
filtering the enlarged image with a nonlinear filter that includes a neural network that inputs modified digitized pixel data that is normalized and based on subtraction of a low-frequency component from element-wise grouped pixels and generates a filtered output that is a sum of the low-frequency component and an inversely normalized output of the neural network to achieve resolution enhancement.
2. The method according to claim 1 , wherein the predetermined frame rate is inversely proportional to the proportion of the optically sensitive part of the digital imaging sensor being scanned.
3. The method according to claim 1 , wherein the enlarged image is formed by a combination of only the sharpest frames of the plurality of low resolution frames.
4. The method according to claim 1 wherein combining the low resolution frames includes selectively averaging the low resolution frames.
5. The method according to claim 4 wherein the low resolution frames are weighted inversely proportional to a noise level of the low resolution frames.
6. A nonlinear digital filter for image super-resolution of an image, input data of which are pixels of the image, the nonlinear digital filter includes a neural network pretrained with a reference image, wherein the neural network receives modified digitized pixel data that is normalized and based on subtraction of a low-frequency component from element-wise grouped pixels, and wherein a final output of the nonlinear digital filter is based on a sum of an output of the neural network that is inversely normalized and the low frequency component.
7. A method according to claim 6 , wherein only the data of the pixel, which has to be filtered, and of those pixels at a distance of not more than 3 pixels, both horizontally and vertically, are used to get the output value of the filter.
8. A method according to claim 6 , wherein the reference image contains line and sinusoidal targets, as well as control points.