System and method for preprocessing of focal planes data for rendering
Systems and methods for rendering a 3D image are provided. The system receives texture and depth data (depth map) for an image. The system generates, based on the image data, a plurality of folded focal planes matrices. For each respective pixel matrix, the system preprocesses pixel values in the respective focal pixel plane matrix to generate a respective preprocessed matrix, wherein the respective preprocessed matrix clusters together pixel values of the respective folded focal plane matrix based on the depth data for the image. The system generates phase functions based on a plurality of the preprocessed matrices. The system configures a spatial light modulator of the SLM device in accordance with the generated phase functions. The system then provides the plurality of the preprocessed matrices as input to the SLM device to generate for display a 3D representation of the received image data.
1. A method comprising:
accessing data for a plurality of pixels for at least two focal planes, wherein the data defined a respective luminance and a respective depth for each pixel on the at least two focal planes;
for each of the at least two focal planes:
mapping respective pixels of the respective focal plane to a respective mapped focal plane such that the respective pixels become clustered based on depth of the respective pixels;
storing mapping data, wherein the mapping data comprises initial and new positions of each mapped pixel of the respective focal plane and a corresponding respective mapped focal plane;
generating a phase function based on the stored mapping data;
generating on a 2D display images based on the respective mapped focal planes for each of the each of the at least two focal planes;
adjusting, by a spatial light modulator (SLM) light produced by the 2D display; and, wherein the adjusting is based on the generated the phase function; and
generating for a display a 3D image based on light adjusted by the SLM.
2. The method of claim 1 , wherein the accessed focal planes comprise a preprocessed set of reduced pixel matrices to fewer focal plane matrices.
3. The method of claim 1 , wherein the data for the plurality of pixels of the at least two focal planes comprises color data for each pixel.
4. The method of claim 3 , prior to the accessing data for the plurality of pixels for the at least two focal planes reducing dynamic range of the plurality of pixels such that chromatic aberration and geometric distortions are reduced.
5. The method of claim 1 , wherein each of the at least two focal planes define image data for different depths ranges.
6. The method of claim 1 , wherein each pair of pixels in same location on the at least two focal planes represents a voxel located between the at least two focal planes by a depth blending algorithm.
7. The method of claim 6 , wherein the voxel that is mapped exactly to a focal plane is displayed by lighting up a single location at a single distance while a voxel that does not map exactly to the focal plane is rendered as a blended pixel using a plurality of adjacent display planes to generate a 3D image.
8. The method of claim 1 , wherein the clustering based on depth of the respective pixels comprises using a machine learning technique that optimizes the cluster maps spatially, so that pixels of depths adjacent to one another are clustered as close to each other as possible.
9. The method of claim 1 , wherein the phase function for each pixel for the SLM is configured to optically reverse changes to the data performed during clustering.
10. The method of claim 1 , wherein the focal planes comprises a display stack with each display corresponding to one of the planes.
11. A system comprising:
control circuitry configured to:
access data for a plurality of pixels for at least two focal planes, wherein the data defined a respective luminance and a respective depth for each pixel on the at least two focal planes;
for each of the at least two focal planes:
map respective pixels of the respective focal plane to a respective mapped focal plane such that the respective pixels become clustered based on depth of the respective pixels;
store mapped data, wherein the mapping data comprises initial and new positions of each mapped pixel of the respective focal plane and a corresponding respective mapped focal plane;
generate a phase function based on the stored mapping data;
generate on a 2D display images based on the respective mapped focal planes for each of the each of the at least two focal planes;
a spatial light modulator (SLM) configured to:
adjust, light produced by the 2D display; and, wherein the adjusting is based on the generated the phase function to generate for a display a 3D image based on light adjusted by the SLM.
12. The system of claim 11 , wherein the control circuitry is configured to access focal planes which comprise a preprocessed set of reduced pixel matrices to fewer focal plane matrices.
13. The system of claim 11 , wherein the data for the plurality of pixels of the at least two focal planes comprises color data for each pixel.
14. The system of claim 13 , wherein prior to the accessing data for the plurality of pixels for the at least two focal planes reducing dynamic range of the plurality of pixels such that chromatic aberration and geometric distortions are reduced.
15. The system of claim 11 , wherein each of the at least two focal planes define image data for different depths ranges.
16. The system of claim 11 , wherein each pair of pixels in same location on the at least two focal planes represents a voxel located between the at least two focal planes by a depth blending algorithm.
17. The system of claim 16 , wherein the voxel that is mapped exactly to a focal plane is displayed by lighting up a single location at a single distance while a voxel that does not map exactly to the focal plane is rendered as a blended pixel using a plurality of adjacent display planes to generate a 3D image.
18. The system of claim 11 , wherein the clustering based on depth of the respective pixels comprises a machine learning technique that optimizes the cluster maps spatially, so that pixels of depths adjacent to one another are clustered as close to each other as possible.
19. The system of claim 11 , wherein the phase function for each pixel for the SLM is configured to optically reverse changes to the data done during clustering.
20. The system of claim 11 , wherein the focal planes comprises a display stack with each display corresponding to one of the planes.