Calibration of light-field camera geometry via robust fitting
Microlens positions for a light-field capture device may be calibrated. A calibration light-field image may be captured, with a microlens portion corresponding to each microlens of the light-field capture device. Interstitial spaces between the microlens portions may be identified and used to locate one or more center locations of the microlens portions. The center locations may be used to generate a model that indicates the microlens positions. Additionally or alternatively, the calibration light field image may be used to select one or more contour samples from among multiple contour samples of the microlens portions. The contour sample may be fitted to a circle centered at a center location of a microlens portion to identify the center location, which may then be used to generate a model that indicates the microlens positions. Multiple iterations may be used to enhance the accuracy of the models.
1. A method for calibrating microlens positions for a light-field image capture device having a plurality of microlenses, the method comprising:
in a data store, receiving a calibration light-field image captured by the light-field image capture device, the calibration light-field image comprising a plurality of microlens portions, each of which comprises microlens light-field data received through a single microlens of the plurality of microlenses;
in a processor, identifying a plurality of interstitial spaces between the microlens portions to obtain a plurality of interstitial space locations; and
in the processor, using the interstitial space locations to generate a model that is indicative of the microlens positions, wherein the model can be applied in generating projected images from light-field data captured by the light-field image capture device.
2. The method of claim 1 , further comprising, in the processor, inverting and eroding the calibration light-field image to generate an inverted and eroded light-field image in which the interstitial spaces have relatively high intensities;
wherein identifying the plurality of interstitial spaces comprises analyzing the inverted and eroded light-field image.
3. The method of claim 1 , wherein using the interstitial space locations to generate the model comprises using at least two of the interstitial space locations adjacent to a first microlens portion of the plurality of microlens portions to identify a first center location of a first center of the first microlens portion.
4. The method of claim 3 , wherein the model comprises a sheared Cartesian grid defined by parameters comprising at least one of the microlens positions, a slope between individual microlenses of the plurality of microlenses, and a spacing between individual microlenses of the plurality of microlenses.
5. The method of claim 3 , further comprising, after generation of the model:
in the processor, identifying a second center location of a second center of a second microlens portion of the plurality of microlens portions; and
in the processor, using the first center location and the second center location to modify the model to generate a modified model that is more accurately indicative of the microlens positions than the model.
6. The method of claim 5 , further comprising, after generation of the modified model:
in the processor, identifying a third center location of a third center of a third microlens portion of the plurality of microlens portions;
in the processor, identifying a fourth center location of a fourth center of a fourth microlens portion of the plurality of microlens portions; and
in the processor, using the third center location and the fourth center location to modify the modified model to generate a further modified model that is more accurately indicative of the microlens positions than the modified model.
7. The method of claim 6 , further comprising:
in the processor, eroding the calibration light-field image to generate an eroded light-field image;
in the processor, using the eroded light-field image to identify a fifth center location of a fifth center of a first microlens portion of the plurality of microlens portions;
in the processor, using the eroded light-field image to identify a sixth center location of a sixth center of a sixth microlens portion of the plurality of microlens portions;
in the processor, using the fifth center location and the sixth center location to generate a preliminary model that is indicative of the microlens positions; and
in the processor, inverting and eroding the calibration light-field image to generate an inverted and eroded light-field image in which the interstitial spaces have relatively high intensities;
wherein identifying the plurality of interstitial spaces comprises analyzing the inverted and eroded light-field image;
wherein each of identifying the first center of the first microlens portion, identifying the second center of the second microlens portion, identifying the third center of the third microlens portion, and identifying the fourth center of the fourth microlens portion comprises using the inverted and eroded light-field image.
8. A method for calibrating microlens positions for a light-field image capture device having a plurality of microlenses, the method comprising:
in a data store, receiving a calibration light-field image captured by the light-field image capture device, the calibration light-field image comprising a plurality of microlens portions, each of which comprises microlens light-field data received through a single microlens of the plurality of microlenses;
in a processor, using the calibration light-field image to select a first contour sample of a first microlens portion of the plurality of microlens portions from among a plurality of contour samples;
in the processor, fitting the first contour sample to a circle centered at a first center location of the first microlens portion to identify the first center location; and
in the processor, using the first center location to generate a model that is indicative of the microlens positions.
9. The method of claim 8 , further comprising, in the processor, prior to using the calibration light-field image to select the first contour sample of the first microlens portion, generating a preliminary model that is indicative of the microlens positions, wherein the model is more accurately indicative of the microlens positions than the preliminary model.
10. The method of claim 9 , wherein using the calibration light-field image to select the first contour sample comprises:
using the calibration light-field image to generate a resampled calibration light-field image with higher angular resolution than the light-field calibration image;
identifying the plurality of contour samples in the resampled calibration light-field image; and
selecting the first contour sample from within the resampled calibration light-field image.
11. The method of claim 10 , wherein using the calibration light-field image to select the first contour sample comprises applying the preliminary model to the resampled calibration light-field image to crop a window from the resampled calibration light-field image, wherein the window contains a majority of the first microlens portion.
12. The method of claim 8 , wherein selecting the first contour sample comprises:
determining that the first contour sample has a strong intensity gradient; and
determining that the first contour sample has an intensity oriented toward an optical center of the calibration light-field image.
13. The method of claim 8 , wherein selecting the first contour sample comprises rejecting outliers of the plurality of contour samples, wherein the outliers are not readily fitted to the circle.
14. The method of claim 13 , wherein rejecting the outliers comprises applying a RANSAC algorithm.
15. A non-transitory computer-readable medium for calibrating microlens positions for a light-field image capture device having a plurality of microlenses, comprising instructions stored thereon, that when executed by a processor, perform the steps of:
causing a data store to receive a calibration light-field image captured by the light-field image capture device, the calibration light-field image comprising a plurality of microlens portions, each of which comprises microlens light-field data received through a single microlens of the plurality of microlenses;
identifying a plurality of interstitial spaces between the microlens portions to obtain a plurality of interstitial space locations; and
using the interstitial space locations to generate a model that is indicative of the microlens positions, wherein the model can be applied in generating projected images from light-field data captured by the light-field image capture device.
16. The non-transitory computer-readable medium of claim 15 , further comprising instructions stored thereon, that when executed by a processor, invert and erode the calibration light-field image to generate an inverted and eroded light-field image in which the interstitial spaces have relatively high intensities;
wherein identifying the plurality of interstitial spaces comprises analyzing the inverted and eroded light-field image.
17. The non-transitory computer-readable medium of claim 15 , wherein using the interstitial space locations to generate the model comprises using at least two of the interstitial space locations adjacent to a first microlens portion of the plurality of microlens portions to identify a first center location of a first center of the first microlens portion.
18. The non-transitory computer-readable medium of claim 17 , wherein the model comprises a sheared Cartesian grid defined by parameters comprising at least one of the microlens positions, a slope between individual microlenses of the plurality of microlenses, and a spacing between individual microlenses of the plurality of microlenses.
19. The non-transitory computer-readable medium of claim 17 , further comprising instructions stored thereon, that when executed by a processor, after generation of the model:
identify a second center location of a second center of a second microlens portion of the plurality of microlens portions; and
use the first center location and the second center location to modify the model to generate a modified model that is more accurately indicative of the microlens positions than the model.
20. A non-transitory computer-readable medium for calibrating microlens positions for a light-field image capture device having a plurality of microlenses, the non-transitory computer-readable medium comprising instructions stored thereon, that when executed by a processor, perform the steps of:
causing a data store to receive a calibration light-field image captured by the light-field image capture device, the calibration light-field image comprising a plurality of microlens portions, each of which comprises microlens light-field data received through a single microlens of the plurality of microlenses;
use the calibration light-field image to select a first contour sample of a first microlens portion of the plurality of microlens portions from among a plurality of contour samples;
fit the first contour sample to a circle centered at a first center location of the first microlens portion to identify the first center location; and
use the first center location to generate a model that is indicative of the microlens positions.
21. The non-transitory computer-readable medium of claim 20 , further comprising instructions stored thereon, that when executed by a processor, prior to using the calibration light-field image to select the first contour sample of the first microlens portion, generate a preliminary model that is indicative of the microlens positions, wherein the model is more accurately indicative of the microlens positions than the preliminary model.
22. The non-transitory computer-readable medium of claim 21 , wherein using the calibration light-field image to select the first contour sample comprises:
using the calibration light-field image to generate a resampled calibration light-field image with higher angular resolution than the light-field calibration image;
identifying the plurality of contour samples in the resampled calibration light-field image;
selecting the first contour sample from within the resampled calibration light-field image; and
applying the preliminary model to the resampled calibration light-field image to crop a window from the resampled calibration light-field image, wherein the window contains a majority of the first microlens portion.
23. The non-transitory computer-readable medium of claim 20 , wherein selecting the first contour sample comprises:
determining that the first contour sample has a strong intensity gradient; and
determining that the first contour sample has an intensity oriented toward an optical center of the calibration light-field image.
24. The non-transitory computer-readable medium of claim 20 , wherein selecting the first contour sample comprises rejecting outliers of the plurality of contour samples, wherein the outliers are not readily fitted to the circle by applying a RANSAC algorithm.
25. A system for calibrating microlens positions for a light-field image capture device having a plurality of microlenses, the system comprising:
a data store configured to receive a calibration light-field image captured by the light-field image capture device, the calibration light-field image comprising a plurality of microlens portions, each of which comprises microlens light-field data received through a single microlens of the plurality of microlenses; and
a processor, communicatively coupled to the data store, configured to:
identify a plurality of interstitial spaces between the microlens portions to obtain a plurality of interstitial space locations; and
use the interstitial space locations to generate a model that is indicative of the microlens positions, wherein the model can be applied in generating projected images from light-field data captured by the light-field image capture device.
26. The system of claim 25 , wherein the processor is further configured to invert and erode the calibration light-field image to generate an inverted and eroded light-field image in which the interstitial spaces have relatively high intensities;
wherein the processor is configured to identify the plurality of interstitial spaces by analyzing the inverted and eroded light-field image.
27. The system of claim 25 , wherein the processor is configured to use the interstitial space locations to generate the model by using at least two of the interstitial space locations adjacent to a first microlens portion of the plurality of microlens portions to identify a first center location of a first center of the first microlens portion.
28. The system of claim 27 , wherein the model comprises a sheared Cartesian grid defined by parameters comprising at least one of the microlens positions, a slope between individual microlenses of the plurality of microlenses, and a spacing between individual microlenses of the plurality of microlenses.
29. The system of claim 27 , wherein the processor is further configured, after generation of the model, to:
identify a second center location of a second center of a second microlens portion of the plurality of microlens portions; and
use the first center location and the second center location to modify the model to generate a modified model that is more accurately indicative of the microlens positions than the model.
30. A system for calibrating microlens positions for a light-field image capture device having a plurality of microlenses, the system comprising:
a data store configured to receive a calibration light-field image captured by the light-field image capture device, the calibration light-field image comprising a plurality of microlens portions, each of which comprises microlens light-field data received through a single microlens of the plurality of microlenses; and
a processor, communicatively coupled to the data store, configured to:
use the calibration light-field image to select a first contour sample of a first microlens portion of the plurality of microlens portions from among a plurality of contour samples;
fit the first contour sample to a circle centered at a first center location of the first microlens portion to identify the first center location; and
use the first center location to generate a model that is indicative of the microlens positions.
31. The system of claim 30 , wherein the processor is further configured, prior to using the calibration light-field image to select the first contour sample of the first microlens portion, to generate a preliminary model that is indicative of the microlens positions;
wherein the model is more accurately indicative of the microlens positions than the preliminary model.
32. The system of claim 31 , wherein the processor is configured to use the calibration light-field image to select the first contour sample by:
using the calibration light-field image to generate a resampled calibration light-field image with higher angular resolution than the light-field calibration image;
identifying the plurality of contour samples in the resampled calibration light-field image;
selecting the first contour sample from within the resampled calibration light-field image; and
applying the preliminary model to the resampled calibration light-field image to crop a window from the resampled calibration light-field image, wherein the window contains a majority of the first microlens portion.
33. The system of claim 30 , wherein the processor is configured to select the first contour sample by:
determining that the first contour sample has a strong intensity gradient; and
determining that the first contour sample has an intensity oriented toward an optical center of the calibration light-field image.
34. The system of claim 30 , wherein the processor is configured to select the first contour sample by rejecting outliers of the plurality of contour samples by applying a RANSAC algorithm, wherein the outliers are not readily fitted to the circle.