Three-dimensional integral-imaging light field display and optimization method therefor

1. An apparatus comprising:

a pixel array comprising a plurality of pixels for receiving-and-displaying or sensing-and-outputting a plurality of elemental images;

a multi-lens array (MLA) coupled to the pixel array, the MLA comprising a plurality of lenslets; and

a circuitry functionally coupled to the pixel array, the circuitry comprising a model for converting a plurality of perspective views to the plurality of elemental images and sending the plurality of elemental images to the pixel array;

wherein the model comprises a deconvolution neural network; and

wherein the model and one or more characteristics of the plurality of lenslets are jointly optimized.

2. The apparatus of claim 1 , wherein the plurality of lenslets are refractive lenslets, diffractive lenslets, or metasurface lenslets.

3. The apparatus of claim 1 , wherein parameters of the deconvolution neural network and the one or more characteristics of the plurality of lenslets are jointly optimized.

4. The apparatus of claim 1 , wherein a plurality of weights of the deconvolution neural network and the one or more characteristics of the plurality of lenslets are jointly optimized.

5. The apparatus of claim 1 , wherein the model and one or more characteristics of the plurality of lenslets are jointly optimized by using a deconvolution neural network model.

6. A method for training the deconvolution neural network model of claim 5 , the method comprising:

converting a set of input perspective images to a set of elemental images;

distorting the set of elemental images by using the deconvolution neural network and point spread functions (PSFs) of the MLA at vicinity θ+ϵ of a plurality of viewing angles θ, the deconvolution neural network comprising one or more first parameters, and the PSFs being generated based on one or more second parameters of the MLA;

adding Gaussian and Poisson display-noise display noise to the set of distorted elemental images to generate synthetic measurement (SM);

using a convolution function to convolve the SM with PSFs of the MLA at the plurality of viewing angles θ to generate a plurality of second perspective images;

comparing the input perspective images and the output perspective images to generate a loss; and

adjusting the one or more first parameters and the one or more second parameters to minimizing the loss.

7. The method of claim 6 , wherein said converting the set of input perspective images to the set of elemental images by using a pixel-mapping algorithm comprises:

converting the set of input perspective images to the set of elemental images by using a pixel-mapping algorithm.

8. A method for training the deconvolution neural network model of claim 5 , the method comprising:

using a convolution function to convolve a plurality of first perspective images with PSFs of the MLA at vicinity θ+ϵ of a plurality of viewing angles θ to generate a set of elemental images, the PSFs being generated based on one or more first parameters of the MLA;

adding Gaussian and Poisson display noise to the set of elemental images to generate SM;

distorting the SM by using a deconvolution neural network and PSFs at the plurality of viewing angles θ to obtain a plurality of distorted perspective images, the deconvolution neural network comprising one or more second parameters;

comparing the first perspective images and the second perspective images to generate a loss; and

adjusting the one or more first parameters and the one or more second parameters to minimizing the loss.

9. A method for evaluating the apparatus of claim 1 , the method comprising:

inputting a plurality of first perspective images to a circuitry of the apparatus for converting the first perspective images to a plurality of elemental images;

displaying the plurality of elemental images through a MLA of the apparatus;

capturing a plurality of second perspective images displayed through the MLA of the apparatus along a plurality of viewing angles; and

comparing the first perspective images and the second perspective images for evaluating the apparatus.