Method and apparatus for estimating depth of field information

1. A method for extracting depth information from a focal stack, comprising: processing the focal stack through a focus convolutional neural network (Focus-Net) comprising a plurality of branches to obtain a plurality of feature maps, each branch comprising a downsampling convolutional layer having a different stride for downsampling the focal stack and a deconvolutional layer for upsampling the focal stack; stacking the plurality of feature maps together; fusing the plurality of feature maps by a plurality of first convolutional layers to obtain a depth image; processing the focal stack through an extended depth-of-field convolutional neural network (EDoF-Net) to obtain an extended depth-of-field (EDoF) image; concatenating the depth image and the EDoF image; and fusing the depth image and the EDoF image using a plurality of third convolutional layers to refine the depth image.

2. The method of claim 1 , wherein the Focus-Net comprises a parametric rectified linear unit (PreLU) layer.

3. The method of claim 1 , wherein the Focus-Net comprises four branches.

4. The method of claim 1 , wherein the EDoF-Net comprises a plurality of second convolutional layers and a PreLU layer.

5. A method for extracting depth information from a stereo image, comprising: processing the stereo image through a stereo convolutional neural network (Stereo-Net) to obtain a second depth image, the Stereo-Net comprising a plurality of rounds of a downsampling part and an upsampling part, wherein each downsampling part comprising a plurality of max pooling layers interleaved with a plurality of first residual modules, each upsampling part comprising a plurality of deconvolutional layers interleaved with a plurality of second residual modules; processing a first focal stack through a focus convolutional neural network (Focus-Net) comprising a plurality of branches to obtain a plurality of feature maps, each branch comprising a downsampling convolutional layer having a different stride to downsample the focal stack and a deconvolutional layer configured to upsample the focal stack; stacking the plurality of feature maps together; fusing the plurality of feature maps by a plurality of first convolutional layers to obtain a first depth image; processing the first focal stack and a second focal stack through an extended depth-of-field convolutional neural network (EDoF-Net) to obtain the stereo image comprising a first EDoF image and a second EDoF image; concatenating the first depth image and the first EDoF image; and fusing the first depth image and the first EDoF image using a plurality of third convolutional layers to refine the first depth image.

6. The method of claim 5 , wherein the downsampling part is connected to the upsampling part through a connection layer comprising a third residue module.

7. The method of claim 5 , wherein the Stereo-Net comprises two rounds of downsampling part and upsampling part.

8. The method of claim 5 , wherein the FocusNet comprises a PreLU layer.

9. The method of claim 5 , wherein the FocusNet comprises four branches.

10. The method of claim 5 , further comprising: processing the first focal stack and a second focal stack through an extended depth-of-field convolutional neural network (EDoF-Net) to obtain the stereo image comprising a first EDoF image and a second EDoF image.

11. The method of claim 5 , further comprising: concatenating the first depth image and the second depth image; and fusing the first depth image and the second depth image using a plurality of fourth convolutional layers to obtain a third depth image.

12. An apparatus for extracting depth information from a focal stack, comprising: a data capture unit comprising a first digital camera configured to generate a first focal stack; and a data processing unit comprising a processor and a memory, the memory embodying instructions that when executed by the processor cause the processor to: process the first focal stack through a focus convolutional neural network (Focus-Net) comprising a plurality of branches to obtain a plurality of feature maps, each branch comprising a downsampling convolutional layer having a different stride for downsampling the focal stack and a deconvolutional layer for upsampling the focal stack; stack the plurality of feature maps together; fuse the plurality of feature maps by a plurality of first convolutional layers to obtain a first depth image; process the first focal stack through an extended depth-of-field convolutional neural network (EDoF-Net) to obtain a first EDoF image, the EDoF-Net comprising a plurality of second convolutional layers and a PreLU layer; concatenate the first depth image and the first EDoF image; and fuse the first depth image and the first EDoF image using a plurality of third convolutional layers to refine the first depth image.

13. The apparatus of claim 12 , wherein the FocusNet comprises four branches.

14. The apparatus of claim 12 , wherein the data capture unit comprises a second digital camera configured to generate a second focal stack.

15. The apparatus of claim 14 , wherein the processor is further configured to:

process the second focal stack through the extended depth-of-field convolutional neural network (EDoF-Net) to obtain a second EDoF image; and

process the first EDoF image and the second EDoF image through a stereo convolutional neural network (Stereo-Net) to obtain a second depth image, the Stereo-Net comprising a plurality of rounds of a downsampling part and an upsampling part,

wherein each downsampling part comprises a plurality of max pooling layers interleaved with a plurality of first residual modules, each upsampling part comprises a plurality of deconvolutional layers interleaved with a plurality of second residual modules.

16. The apparatus of claim 15 , wherein the downsampling part is connected to the upsampling part through a connection layer comprising a third residue module.

17. The apparatus of claim 16 , wherein the Stereo-Net comprises two rounds of downsampling part and upsampling part.

18. The apparatus of claim 16 , wherein the processor is further configured to:

concatenate the first depth image and the second depth image; and

fuse the first depth image and the second depth image using a plurality of fourth convolutional layers to obtain a third depth image.