Range measurement using a zoom camera
View Patent ↗A method of using an image capture device to identify range information for objects in a scene, includes providing an image capture device having at least one image sensor, a lens and a coded aperture, storing in a memory a set of blur kernels derived from range calibration data for the coded aperture, capturing a first and second image of the scene having a plurality of objects, corresponding to first and second optical magnifications, respectively. The method further includes providing a set of deblurred images using the capture images from each magnification and each of the blur kernels from the stored set, and using the set of deblurred images to determine the range information for the objects in the scene.
1. A method of using an image capture device to identify range information for objects in a scene, comprising:
a) providing an image capture device having at least one image sensor, a lens and a coded aperture;
b) storing in a memory a set of blur kernels derived from range calibration data for the coded aperture;
c) capturing a first and second image of the scene having a plurality of objects, corresponding to first and second optical magnifications, respectively, wherein the image capture device has a different spatial frequency response at the first and second optical magnifications;
d) providing a set of deblurred images using the capture images from each magnification and each of the blur kernels from the stored set; and
e) using the set of deblurred images to determine the range information for the objects in the scene.
2. The method of claim 1 wherein step d) includes for each deblurred image
i) initializing a candidate deblurred image;
ii) determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image, including a range of at least 1 or more pixels in size and including vertical, horizontal and diagonal directions;
iii) determining a combined differential image by combining the differential images;
iv) updating the candidate deblurred image responsive to the captured image, the blur kernels, the candidate deblurred image and the combined differential image; and
v) repeating steps i)-iv) until a convergence criterion is satisfied.
3. The method of claim 1 , wherein step c) includes capturing first and second image sequences, corresponding to the first and second optical magnifications, respectively.
4. The method of claim 3 , wherein step e) includes determining range information for each image in the sequence.
5. The method of claim 4 , wherein range information is determined for a subset of images in the sequence.
6. The method of claim 4 , wherein the range information is used to identify stationary and moving objects in the scene.
7. The method of claim 6 , wherein the range information is used by the image capture device to track moving objects.
8. The method of claim 3 , wherein the step of initializing a candidate deblurred image includes:
a) determining a difference image between the current and previous image in the image sequence; and
b) initializing a candidate deblurred image responsive to the difference image.
9. The method of claim 8 , further including determining range information for the objects in the scene, responsive to the difference image.
10. The method of claim 1 , wherein step d) includes using a subset of blur kernels from the stored set.
11. The method of claim 1 , wherein step b) includes using a set of blur parameters derived from calibration data at a set of range values, such that there is a set of blur parameters for each optical magnification at each corresponding range value.
12. The method of claim 1 , wherein step b) includes storing a subset of blur parameters derived from calibration data at a set of range values, responsive to information provided by the capture device.
13. The method of claim 1 , wherein step e) includes combining the deblurred images according to a spatial-frequency dependent weighting criterion.
14. The method of claim 1 , wherein step e) includes combining deblurred images resulting from blur parameters corresponding to each optical magnification within selected range intervals.
15. The method of claim 14 , further including combining the deblurred images according to a spatial-frequency dependent weighting criterion.
16. A digital camera system comprising:
a) an image sensor for capturing one or more images of a scene;
b) a zoom lens for imaging the scene onto the image sensor at first and second optical magnifications, wherein the image has a different spatial frequency response at first and second optical magnifications;
c) a coded aperture;
d) a processor-accessible memory for storing a set of blur parameters derived from range calibration data; and
e) a data processing system for providing a set of deblurred images using captured images and each of the blur parameters from the stored set by,
i) initializing a candidate deblurred image;
ii) determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image, including a range of at least 1 or more pixels in size and including vertical, horizontal and diagonal directions;
iii) determining a combined differential image by combining the differential images;
iv) updating the candidate deblurred image responsive to the captured image, the blur kernel, the candidate deblurred image and the combined differential image;
v) repeating steps i)-iv) until a convergence criterion is satisfied; and
vi) using the set of deblurred images to determine the range information for the objects in the scene.
17. A method of using an image capture device to identify range information for objects in a scene, comprising:
a) providing an image capture device having at least one image sensor, a lens and a coded aperture;
b) storing in a memory a set of blur kernels derived from range calibration data for the coded aperture, using a set of blur parameters derived from calibration data at a set of range values, wherein there is a set of blur parameters for each optical magnification at each corresponding range value, and wherein the sets of blur parameters corresponding to each optical magnification are associated with overlapping range intervals;
c) capturing a first and second image of the scene having a plurality of objects, corresponding to first and second optical magnifications, respectively, wherein the scene's spatial frequency content is filtered by the image capture device's frequency response at a different scale at the two different magnifications;
d) providing a set of deblurred images using the capture images from each magnification and each of the blur kernels from the stored set; and
e) using the set of deblurred images to determine the range information for the objects in the scene.
18. A method of using an image capture device to identify range information for objects in a scene, comprising:
a) providing an image capture device having at least one image sensor, a lens and a coded aperture;
b) storing in a memory a set of blur kernels derived from range calibration data for the coded aperture, using a set of blur parameters derived from calibration data at a set of range values, wherein there is a set of blur parameters for each optical magnification at each corresponding range value, and wherein the sets of blur parameters corresponding to each optical magnification are associated with non-overlapping range intervals;
c) capturing a first and second image of the scene having a plurality of objects, corresponding to first and second optical magnifications, respectively, wherein the scene's spatial frequency content is filtered by the image capture device's frequency response at a different scale at the two different magnifications;
d) providing a set of deblurred images using the capture images from each magnification and each of the blur kernels from the stored set; and
e) using the set of deblurred images to determine the range information for the objects in the scene.