Method and system for acquiring, encoding, decoding and displaying 3D light fields

1. A computer implemented method for acquiring and displaying light fields, comprising, the steps of:

interpolating input samples of an input light field to produce a reparameterized light field, in which the input samples are different views acquired of a three dimensional scene by a plurality of cameras, and in which the interpolating is according to an acquisition parameterization and a display parameterization;

prefiltering the reparameterized light field to produce a prefiltered light field; and

sampling the prefiltered light field to produce output samples having the display parameterization, the output samples for display as an output light field on a three dimensional display device, in which the sampling is an oversampling in a plane of the three dimensional display device to reduce inter-perspective aliasing in an oversampled light field, and in which the oversampled light field is subsampled according to the display the output samples, and wherein a bandwidth of the three dimensional display device is parameterized according to an angular frequency of the three dimensional display device, and a spatial frequencies of the three dimensional display device.

2. The method of claim 1 , in which the interpolating uses a depth map acquired of the scene.

3. The method of claim 2 , in which the interpolating further comprises:

reprojecting the input samples using an unstructured lumigraph rendering process.

4. The method of claim 1 , in which the interpolating generates additional views of the three dimensional scene.

5. The method of claim 1 , in which the oversampling is by a factor of (d+1)/2, where d is a maximum disparity of the different views.

6. The method of claim 1 , further comprising:

encoding the output light field to produce a compressed bitstream.

7. The method of claim 6 , in which the compressed bitstream is scalable, and further comprising:

decoding the scalable bitstream in a decoder according to the display parameterization to produce a decoded light field; and

resampling the decoded light field according to the display parameterization to produce a sampled light field for the three dimensional display device.

8. The method of claim 7 , in which the scalable decoder determines a target views, and a target spatial resolution for each target view of the decoded light field.

9. The method of claim 8 , in which the target views have temporal dependencies and spatial dependencies, and further comprising:

discarding portions of the compressed bitstream corresponding to selected views during the decoding.

10. The method of claim 8 , in which the target views are a set of views with minimum baseline distance between spatially adjacent views of the plurality of cameras.

11. The method of claim 8 , in which target views have equal baseline distances between spatially adjacent views.

12. The method of claim 8 , in which the target views have temporal dependencies and spatial dependencies, and further comprising:

discarding portions of the compressed bitstream corresponding to a higher spatial resolution than the target spatial resolution during the decoding.

13. The method of claim 7 , in which the decoded light field has a spatial resolution higher than a spatial resolution of the three dimensional display device.

14. The method of claim 6 , in which the encoding determines a maximum disparity between the target views, and further comprising:

signaling the maximum disparity to the decoder.

15. The method of claim 14 , in which the maximum disparity information is signaled in supplemental enhancement information messages according to a H.264/AVC video coding standard.

16. The method of claim 14 , in which the maximum disparity is expressed at a pixel resolution.

17. The method of claim 14 , in which the maximum disparity is expressed at a sub-pixel resolution.

18. The method of claim 1 , in which the interpolating uses a depth map acquired of the scene, and further comprising:

encoding the output light field to produce a compressed bitstream; and

encoding the depth map to produce a depth bitstream.

19. The method of claim 18 , further comprising:

providing the compressed bitstream to a scalable decoder to produce a decoded output light field and providing the depth bitstream to a depth decoder to produce a decoded depth map.

20. The method of claim 18 in which the compressed bitstream and depth bitstream are multiplexed to produce a multiplexed bitstream.

21. The method of claim 20 , further comprising:

providing the multiplexed bitstream to a scalable decoder to produce a decoded output light field and a decoded depth map.

22. The method of claim 21 , in which the decoded output light field includes additional views.

23. The method of claim 22 , in which the additional views are generated using the decoded depth map and view interpolation during the decoding.

24. A system for acquiring and displaying light fields, comprising:

means for interpolating input samples of an input light field to produce a reparameterized light field, in which the input samples are different views acquired of a three dimensional scene by a plurality of cameras, and in which the interpolating is according to an acquisition parameterization and a display parameterization;

means for prefiltering the reparameterized light field to produce a prefiltered light field; and

means for sampling the prefiltered light field to produce output samples having the display parameterization, the output samples for display on an output light field using a three dimensional display device, in which the sampling is an oversampling in a plane of the three dimensional display device to reduce inter-perspective aliasing in an oversampled light field, and in which the oversampled light field is subsampled according to the display parameterization to produce the output samples, and wherein a bandwidth of the three dimensional display device is parameterized according to an angular frequency of the three dimensional display device, and a spatial frequencies of the three dimensional display device.