Method and apparatus for deblurring images
View Patent ↗A method and system deblurs images acquired of a scene by a camera. A light field acquired of a scene is modulated temporally according to a sequence of ons and offs. The modulated light field is integrated by a sensor of a camera during an exposure time to generate an encoded input image. The encoded input image is decoded according to a pseudo-inverse of a smearing matrix to produce a decoded output image having a reduced blur.
1. A method for reducing blur in an image of a scene acquired by a camera, comprising the steps of:
temporally modulating light of a scene acquired and integrated by a camera image sensor according to a temporally coded pattern during an exposure time to generate an encoded input image, in which the camera image sensor is a solid state sensor and includes an array of pixels, and in which the modulating results in a well-posed invertible Fourier transform of a resultant point spread function; and
decoding the encoded input image according to an inverse of the coded pattern to produce a decoded output image having a reduced blur.
2. The method of claim 1 , in which the blur is caused by camera motion.
3. The method of claim 1 , in which the blur is caused by object motion.
4. The method of claim 1 , in which the blur is caused by a combination of camera and object motion.
5. The method of claim 1 , in which the modulating is discrete.
6. The method of claim 1 , in which the modulating is continuous.
7. The method of claim 1 , in which the modulating is selected from the group comprising: amplitude modulation, frequency modulation, phase modulation, pulse width modulation, opacity modulation, polarization modulation and combinations thereof.
8. The method of claim 1 , in which the modulating is performed by repeatedly opening and closing a camera shutter during the exposure time.
9. The method of claim 1 , in which the modulating is performed by repeatedly changing an opacity of a camera shutter during the exposure time.
10. The method of claim 1 , in which the modulating is performed by repeatedly turning integration by the camera image sensor on and off during the exposure time.
11. The method of claim 1 , in which the modulating is performed by repeatedly turning a flash unit illuminating the scene on and off while keeping a camera shutter open during the exposure time.
12. The method of claim 1 , in which the modulating is performed by both repeatedly opening and closing a camera shutter and repeatedly turning a flash unit on and off during the exposure time.
13. The method of claim 1 , in which the camera image sensor is a single photosensor.
14. The method of claim 1 , in which the camera is a video camera configured to generate a sequence of input images and a sequence of deblurred output images.
15. The method of claim 14 , in which different temporal modulating is used for different images in the sequence.
16. The method of claim 1 , in which the modulating maximizes a coverage in a Fourier domain of a resultant point spread function.
17. The method of claim 16 , in which the point spread function is an integration of the modulated light and spatial motion of a moving object in the scene.
18. The method of claim 1 , in which the decoding is performed by a process operating in the camera.
19. The method of claim 1 , in which the temporal modulation is selected to preserve relatively higher spatial frequencies of a moving object in the scene and to recover the relatively higher spatial frequencies during the decoding.
20. The method of claim 1 , in which the modulating is according to a pseudo-random sequence of opens and closes of a camera shutter.
21. The method of claim 1 , in which the decoding is performed by a deconvolution process.
22. The method of claim 1 , in which the decoding is performed by a linear algebra process using a least-square estimation to estimate the output image.
23. An apparatus for deblurring an image of a scene including a moving object, comprising:
means for temporally modulating a light field of a scene including a moving object acquired by a sensor of a camera during an exposure time to generate an encoded input image, wherein the modulating is according to a binary on and off sequence of the light field, in which the sensor is a solid state sensor and includes an array of pixels, and in which the modulating results in a well-posed invertible Fourier transform of a resultant point spread function; and
means for decoding the encoded input image to produce to a decoded output image having a reduced blur.
24. The apparatus of claim 23 , in which the means for modulating is a shutter of the camera.
25. The apparatus of claim 23 , in which the means for modulating is an opacity filter of the camera.
26. The apparatus of claim 23 , in which the means for modulating is a flash unit of the camera.
27. The apparatus of claim 23 , in which the means for modulating turns the sensor on and off while acquiring the light field.
28. The apparatus of claim 23 , in which the means for modulating modulates the light field in such a way that a resultant point spread function has a maximum coverage in a Fourier domain.
29. The apparatus of claim 23 , in which the input image is a convolution between a temporal integration of the modulated light field and spatial object motion.
30. The apparatus of claim 23 , in which the modulating is continuous.
31. The apparatus of claim 23 , in which the modulating is discrete.
32. The apparatus of claim 23 , in which the binary on and off sequence is
1010000111000001010000110011110111010111001001100111.
33. The apparatus of claim 23 , in which a motion of the camera is recovered.
34. A method for reducing blur in an image of a scene, comprising the steps of:
temporally modulating an energy field of a scene acquired and integrated by a sensor according to a temporally coded pattern during an exposure time to generate an encoded input image, in which the camera image sensor is a solid state sensor and includes an array of pixels, and in which the modulating results in a well-posed invertible Fourier transform of a resultant point spread function; and
decoding the encoded input image according to an inverse of the coded pattern to produce a decoded output image having a reduced blur.
35. The method of claim 34 , in which the energy field is electromagnetic radiation.
36. The method of claim 34 , in which the energy field is acoustic radiation.
37. The method of claim 34 , in which the modulating is continuous.
38. The method of claim 34 , in which the modulating is discrete.
39. The method of claim 34 , in which the modulation is selected from the group consisting of: amplitude modulation, frequency modulation, phase modulation, pulse width modulation, opacity modulation, polarization modulation and combinations thereof of a shutter between the scene and the sensor.