Method of applying atmospheric turbulence disturbances to images
An image generator engine for generating distorted images from an input image is disclosed which includes a random seed generator adapted to generate random seeds, a model adapted to receive the random seeds and generate a first plurality of sets of Zernike coefficients (FPSZC), a pixel shifter adapted to shift pixels of the input image, a phase to space (P2S) transformer adapted to in parallel receive the FPSZC and generate a predetermined number of P2S coefficients, a learned basis function generator adapted to generate learned basis function matrices, a convolver adapted to convolve each pixel of the tilted image by the learned basis functions thereby generating a plurality of convolved pixels, and a blurrer adapted to receive the plurality of the convolved pixels and multiply by the corresponding P2S coefficients and apply a summation to generate the distorted image.
1. An image generator engine for generating distorted images from an input image, comprising:
a random seed generator adapted to generate a first plurality of sets of random seeds (FPSRS), the number of sets of random seeds corresponds to the number of pixels in the input image;
an atmospheric disturbance model adapted to receive the FPSRS and in response thereto generate a first plurality of sets of Zernike coefficients (FPSZC), each set of Zernike coefficients of the FPSZC corresponding to an associated set of random seeds of the FPSRS, and each set of Zernike coefficients of the FPSRS and each set of random seeds of the FPSRS comprising a plurality of corresponding entries;
a pixel shifter adapted to receive in parallel a subset of the entries of the FPSZC and the input image, and in response thereto shift the corresponding pixel of the input image to thereby generate a tilted image;
a phase to space (P2S) transformer adapted to in parallel receive remainder of entries (RoE) of the FPSZC and in response generate a predetermined number of P2S coefficients (PNP2SC) defining a non-linear mapping of the plurality of sets of Zernike coefficients to associated point spread functions (PSFs) basis coefficients;
a learned basis function generator adapted to receive tilt-free PSFs from a plurality of PSFs generated corresponding to a second plurality of sets of Zernike coefficients (SPSZC) generated from the atmospheric disturbance model receiving a second plurality of sets of random seeds (SPSRS), wherein each set of Zernike coefficients of the SPSZC corresponding to an associated set of random seeds of the SPSRS, and each set of Zernike coefficients of the SPSZC and each set of random seeds of the SPSRS comprising a plurality of corresponding entries and perform a principal component analysis on the tilt-free PSFs, thereby generating the learned basis functions represented as PNP2SC matrices;
a convolver adapted to convolve each pixel of the tilted image by the learned basis functions thereby generating a plurality of convolved pixels; and
a blurrer adapted to receive the plurality of the convolved pixels and multiply by the corresponding P2S coefficients and apply a summation to generate the distorted image.
2. The image generator engine of claim 1 , wherein the number of entries in each set of the first plurality of sets of Zernike coefficients includes 36 entries.
3. The image generator engine of claim 1 , wherein the subset of the entries of the FPSZC includes 2 subsets.
4. The image generator engine of claim 3 , wherein the P2S transformer is a neural network.
5. The image generator engine of claim 4 , the neural network adapted to receive RoE of the SPSZC and outputs PNP2SC interim P2S coefficients during an optimization process, whereby each of the PNP2SC interim P2S coefficients is multiplied by a corresponding learned basis function matrix, and the resulting matrices added together to generate a neural network generated tilt-free PSF which is compared with a corresponding predetermined tilt-free PSF to generate an error signal which is minimized.
6. The image generator engine of claim 5 , wherein the error minimization is by least squares.
7. The image generator engine of claim 5 , wherein the P2S transformer is the neural network after minimization of the error signal.
8. The image generator engine of claim 4 , wherein the neural network includes RoE inputs, PNP2SC outputs and three fully connected layers.
9. The image generator engine of claim 1 , wherein the number of sets in the FPSZC is m×n, where m is number of pixels in the X-direction and n is number of pixels in the Y-direction.
10. The image generator engine of claim 1 , wherein the number of sets in the FPSRS is m×n, where m is number of pixels in the X-direction and n is number of pixels in the Y-direction.
11. A method of generating distorted images from an input image, comprising:
generating a first plurality of sets of random seeds (FPSRS), the number of sets of random seeds corresponds to the number of pixels in the input image;
modeling atmospheric disturbances by applying an atmospheric disturbance model adapted to receive the FPSRS and in response thereto generate a first plurality of sets of Zernike coefficients (FPSZC), each set of Zernike coefficients of the FPSZC corresponding to an associated set of random seeds of the FPSRS, and each set of Zernike coefficients of the FPSRS and each set of random seeds of the FPSRS comprising a plurality of corresponding entries;
shifting pixels in the input image by a pixel shifter adapted to receive in parallel a subset of the entries of the FPSZC and the input image, and in response thereto shift the corresponding pixel of the input image to thereby generate a tilted image;
applying a phase to space (P2S) transformer adapted to in parallel receive remainder of entries (RoE) of the FPSZC and in response generate a predetermined number of P2S coefficients (PNP2SC) defining a non-linear mapping of the plurality of sets of Zernike coefficients to associated point spread functions (PSFs) basis coefficients;
generating a learned basis function by a learned basis function generator adapted to receive tilt-free PSFs from a plurality of PSFs generated corresponding to a second plurality of sets of Zernike coefficients (SPSZC) generated from the atmospheric disturbance model receiving a second plurality of sets of random seeds (SPSRS), wherein each set of Zernike coefficients of the SPSZC corresponding to an associated set of random seeds of the SPSRS, and each set of Zernike coefficients of the SPSZC and each set of random seeds of the SPSRS comprising a plurality of corresponding entries and perform a principal component analysis on the tilt-free PSFs, thereby generating the learned basis functions represented as PNP2SC matrices;
convolving by a convolver each pixel of the tilted image by the learned basis functions thereby generating a plurality of convolved pixels; and
applying a blurrer adapted to receive the plurality of the convolved pixels and multiply by the corresponding P2S coefficients and apply a summation to generate the distorted image.
12. The method of claim 11 , wherein the number of entries in each set of the first plurality of sets of Zernike coefficients includes 36 entries.
13. The method of claim 11 , wherein the subset of the entries of the FPSZC includes 2 sub sets.
14. The method of claim 13 , wherein the P2S transformer is a neural network.
15. The method of claim 14 , the neural network adapted to receive RoE of the SPSZC and outputs PNP2SC interim P2S coefficients during an optimization process, whereby each of the PNP2SC interim P2S coefficients is multiplied by a corresponding learned basis function matrix, and the resulting matrices added together to generate a neural network generated tilt-free PSF which is compared with a corresponding predetermined tilt-free PSF to generate an error signal which is minimized.
16. The method of claim 15 , wherein the error minimization is by least squares.
17. The method of claim 15 , wherein the P2S transformer is the neural network after minimization of the error signal.
18. The method of claim 14 , wherein the neural network includes RoE inputs, PNP2SC outputs and three fully connected layers.
19. The method of claim 11 , wherein the number of sets in the FPSZC is m×n, where m is number of pixels in the X-direction and n is number of pixels in the Y-direction.
20. The method of claim 11 , wherein the number of sets in the FPSRS is m×n, where m is number of pixels in the X-direction and n is number of pixels in the Y-direction.