Image processing for turbulence compensation

1. A method of compensating image data for phase fluctuations caused by a wave deforming medium, the method comprising:

capturing, by a sensor of an imaging system, first image data and second image data for each of a plurality of pixel positions of the sensor, the sensor capturing an object through a wave deforming medium causing a defocus disparity between the first image data and second image data, the first image data and second image data are captured from different viewpoints;

determining the defocus disparity between the first image data and the second image data, the defocus disparity corresponding to a defocus wavefront deviation of the wave deforming medium; and

compensating the image data captured by the sensor for phase fluctuations caused by the wave deforming medium using the determined defocus disparity.

2. The method according to claim 1 , wherein the first image data and the second image data is captured using a dual-pixel autofocus sensor.

3. The method according to claim 2 , wherein the defocus disparity between the first image data and the second image data relates to displacement between left pixel data and right pixel data of the dual-pixel autofocus sensor.

4. The method according to claim 1 , wherein the first image data and the second image data is a captured by a stereo camera, and the defocus disparity between the first image data and the second image data relates to displacement between left and right image data captured by the stereo camera.

5. The method according to claim 1 , wherein the first image data and the second image data comprise a first image and a second image captured at different times, and the defocus disparity between the first image data and the second image data relates to a relative blur between the first image and the second image.

6. The method according to claim 5 , wherein the defocus wavefront deviation is determined based on a defocus distance, the defocus distance being determined using the relative blur between the first image and the second image.

7. The method according to claim 1 , wherein processing at least one of the first image data and second image data comprises:

determining a look-up table comprising a mapping between a strength of phase fluctuations caused by the wave deforming medium and a tile size for a tile-based turbulence compensation method;

selecting a tile size from the look-up table based on the determined strength of phase fluctuations; and

applying the tile-based compensation method using the selected tile size to correct for the phase fluctuations.

8. The method according to claim 1 , wherein the received first image data and second image data comprises a plurality of frames.

9. The method according to claim 8 , wherein strength of phase fluctuations in a region of a frame is determined based on a plurality of samples of the strength of phase fluctuations determined within the region of the frame.

10. The method according to claim 8 , further comprising:

compensating for phase fluctuations caused by the wave deforming medium based on comparison of the strength of phase fluctuations associated with the plurality of frames.

11. The method according to claim 8 , further comprising:

compensating for phase fluctuations caused by the wave deforming medium by fusing the plurality of frames based on values of the strength of phase fluctuations determined within each one of the plurality of frames.

12. The method according to claim 11 , wherein the fusing comprises:

for each region in a fused image, determining a plurality of corresponding regions from the plurality of frames, each corresponding region being associated with a strength of phase fluctuations;

comparing the strength of phase fluctuations for the determined corresponding regions; and

forming a region of the fused image based on the comparison.

13. The method according to claim 12 , wherein forming a region of the fused image comprises selecting a region from the determined plurality of corresponding regions having a strength of phase fluctuation below a predetermined threshold.

14. The method according to claim 12 , wherein forming a region of the fused image comprises selecting a region from the determined plurality of corresponding regions having a lowest strength of phase fluctuation.

15. The method according to claim 1 , wherein processing at least one of the first image data and second image data comprises deconvolving the image data using a point spread function, the size of the point spread function at a particular position being determined using the strength of phase fluctuation determined at the particular pixel position.

16. The method according to claim 1 , wherein the defocus wavefront deviation is determined based on disparity in the first image data and the second image data with respect to reference image data at a predetermined pixel position, and the reference image data is determined by convolving the first image data with a kernel having a predetermined width.

17. The method according to claim 16 , wherein the disparity is a ratio of a gradient magnitude of the second image data over a gradient magnitude of the reference image data, the ratio determined at a pixel with maximum gradient across an edge.

18. The method according to claim 1 , wherein determining the defocus wavefront deviation comprises estimating the wavefront deviation using one dimensional signals captured using an autofocus sensor of a device capturing the image.

19. The method according to claim 1 , wherein processing at least one of the first image data and second image data comprises:

determining a region associated with the object in a plurality of frames of the received image;

determining an average strength of phase fluctuations for the region in each of the plurality of frames, and

selecting a frame based on the average strength of phase fluctuations for the region.

20. The method according to claim 19 , further comprising generating a fused high-resolution image from the regions based on the average strength of phase fluctuations.

21. The method according to claim 1 , further comprising determining a strength of phase fluctuations caused by the wave deforming medium using the defocus wavefront deviation and lens intrinsic characteristics, the strength of phase fluctuations being determined with reference to a defocus Zernike coefficient.

22. A non-transitory computer readable medium having at least one computer program stored thereon for causing at least one processor to perform a method for determining a turbulence strength for processing image data, the method comprising:

receiving image data for a portion of an image captured by a dual-pixel sensor through a wave deforming medium, the dual pixel data are captured from different viewpoints;

determining a defocus disparity between left pixel data and right pixel data of the dual-pixel sensor, the defocus disparity corresponding to a defocus wavefront deviation of the wave deforming medium; and

determining the turbulence strength caused by the wave deforming medium using the determined defocus disparity between left pixel data and right pixel data to process the image data.

23. An image capture apparatus configured to determine phase fluctuations of a wave deforming medium, the image capturing apparatus comprising:

a memory;

a lens system focusing light travelling from an imaging scene through the wave deforming medium on an image sensor;

the image sensor configured to capture image data from the lens system as first pixel data and second pixel data for each of a plurality of pixel positions, the image sensor being coupled to the memory, the memory storing the captured first pixel data and second pixel data, the first image data and second image data are captured from different viewpoints; and

a processor coupled to the memory and configured to determine phase fluctuations caused by the wave deforming medium using a defocus disparity between the first pixel data and the second pixel data captured by the image sensor.

24. A system, comprising:

an image capture sensor,

a memory, and

a processor, wherein the processor executes code stored on the memory to:

receive, from the image capture sensor, first image data and second image data for each of a plurality of pixel positions of the image capture sensor, the image data capturing an object through a wave deforming medium causing a defocus disparity between the first image data and second image data, the first image data and second image data are captured from different viewpoints;

determine the defocus disparity between the first image data and the second image data, the defocus disparity corresponding to a defocus wavefront deviation of the wave deforming medium; and

compensate the image data captured by the image capture sensor for phase fluctuations caused by the wave deforming medium using the determined defocus disparity.

25. A method of compensating image data for phase fluctuations caused by a wave deforming medium, the method comprising:

capturing, by a sensor of an imaging system, first image data and second image data for each of a plurality of pixel positions of the sensor, the sensor capturing an object through a wave deforming medium causing a defocus disparity between the first image data and second image data;

determining the defocus disparity between the first image data and the second image data, the defocus disparity corresponding to a defocus wavefront deviation of the wave deforming medium; and

compensating the image data captured by the sensor for phase fluctuations caused by the wave deforming medium using the determined defocus disparity, the compensating comprising processing at least one of the first image data and second image data by:

determining a look-up table comprising a mapping between a strength of phase fluctuations caused by the wave deforming medium and a tile size for a tile-based turbulence compensation method;

selecting a tile size from the look-up table based on the determined strength of phase fluctuations; and

applying the tile-based compensation method using the selected tile size to correct for the phase fluctuations.