Optimization of optical systems for improved light field capture and manipulation
According to various embodiments of the present invention, the optical systems of light field capture devices are optimized so as to improve captured light field image data. Optimizing optical systems of light field capture devices can result in captured light field image data (both still and video) that is cheaper and/or easier to process. Optical systems can be optimized to yield improved quality or resolution when using cheaper processing approaches whose computational costs fit within various processing and/or resource constraints. As such, the optical systems of light field cameras can be optimized to reduce size and/or cost and/or increase the quality of such optical systems.
1. In a light field image capture device, the light field image capture device including an aperture, a main lens, a microlens array, at least one sensor, and processing circuitry, the lenses on the microlens array arranged in a regular pattern, the pixels on the at least one sensor arranged in a regular pattern, the microlens array and at least one sensor interoperating to capture light field image data from light passing through the aperture and the main lens, a method for perturbing at least one regular pattern of the light field image capture device to increase the quality of captured light field image data, the method comprising:
accessing light that has passed through the aperture and the main leans;
at the processing circuitry, introducing random variations into at least one of: the regular pattern of lenses in the microlens array and the regular pattern of pixels in the at least one sensor;
subsequent to introduction of the random variations, capturing light field image data from the accessed light; and
sending the captured light field image data to the processing circuitry.
2. The method of claim 1 , wherein introducing random variations into at least one of: the regular pattern of lenses and the regular pattern of pixels comprises introducing a jitter pattern into at least one of: the regular pattern of lenses and the regular pattern of pixels.
3. The method of claim 2 , wherein introducing a jitter pattern into at least one of the regular pattern of lenses and the regular pattern of pixels comprises introducing a jitter pattern into the regular pattern of lenses that extends across the entire microlens array.
4. The method of claim 2 , wherein introducing a jitter pattern into at least one of the regular pattern of lenses and the regular pattern of pixels comprises introducing a jitter pattern into the regular pattern of pixels that extends across the entirety of at least one sensor.
5. The method of claim 2 , wherein introducing a jitter pattern into at least one of the regular pattern of lenses and the regular pattern of pixels comprises introducing a jitter pattern tile into the regular pattern of lenses, the jitter pattern tile being repeated a plurality of times across the entire microlens array.
6. The method of claim 2 , wherein introducing a jitter pattern into at least one of the regular pattern of lenses and the regular pattern of pixels comprises introducing a jitter pattern tile into the regular pattern of pixels, the jitter pattern tile being repeated a plurality of times across the entire at least one sensor.
7. A non-transitory computer program product for use in a light field image capture device, the light field image capture device including an aperture, a main lens, a microlens array, at least one sensor, and processing circuitry, the lenses on the microlens array arranged in a regular pattern, the pixels on the at least one sensor arranged in a regular pattern, the microlens array and at least one sensor interoperating to capture light field image data from light passing through the aperture and the main lens, the non-transitory computer program product comprising instructions stored thereon, that when executed on the processing circuitry of the light field image capture device, perform the steps of:
accessing light that has passed through the aperture and the main leans;
introducing random variations into at least one of: the regular pattern of lenses in the microlens array and the regular pattern of pixels in the at least one sensor;
subsequent to introduction of the random variations, causing the sensor to capture light field image data from the accessed light; and
receiving the captured light field image data at the processing circuitry.
8. A light field image capture device, comprising:
an aperture;
a main lens;
a microlens array having lenses arranged in a regular pattern;
at least one sensor having pixels arranged in a regular pattern, the at least one sensor configured to capture light field image data representing output of the microlens array, the microlens array and the at least one sensor interoperating to capture light field image data from light passing through the aperture and the main lens:
a pre-processor, configured to access light that has passed through the aperture and the main leans and to introduce random variations into at least one of: the regular pattern of lenses in the microlens array and the regular pattern of pixels in the at least one sensor;
processing circuitry, configured to, subsequent to introduction of the random variations, capture light field image data from the accessed light and process the pre-processed light field image data to generate visual output.