Method and system for rendering an image from a light-field camera

1. A method for rendering an image from a light-field camera, the light-field camera generating a raw light-field image in response to detecting light rays incident on a focal plane using a photosensor which detects the light rays from a scene, the light rays being incident on an array of microlenses and passed to different locations on the focal plane, the raw light-field image being composed of a plurality of micro-images, the method to be implemented by an image rendering system and comprising:

(A) generating, by the image rendering system, feature data, which includes feature elements associated with corresponding position information, associated with the raw light-field image, and obtained based on the raw light-field image and a preset threshold condition;

(B) generating, by the image rendering system, a raw focused image from the raw light-field image;

(C) obtaining, by the image rendering system, a virtual focus position that is designated on the raw focused image; and

(D) refocusing, by the image rendering system, the raw focused image according to the virtual focus position by updating pixel values of pixels of the raw focused image that correspond respectively in position to the feature elements of the feature data, so as to generate a refocused image;

wherein a first portion of pixels of the refocused image that correspond respectively in position to the feature elements of the feature data has pixel values which are determined by performing a refocusing operation upon the raw light-field image according to the virtual focus position; and

wherein a second portion of the pixels of the refocused image that do not correspond in position to the feature elements of the feature data has pixel values which are determined according to pixel values of pixels of the raw focused image that correspond respectively in position to the second portion of the pixels of the refocused image.

2. The method according to claim 1 , wherein the second portion of the pixels of the refocused image has pixel values equal to the pixel values of the pixels of the raw focused image that correspond respectively in position to the second portion of the pixels of the refocused image.

3. The method according to claim 1 , wherein step (A) includes the sub-steps of:

(A1) obtaining, by the image rendering system, pixel values of central pixels of the micro-images of the raw light-field image, and generating a full depth-of-field image based on the pixel values thus obtained; and

(A2) performing, by the image rendering system, a gradient operation upon the full depth-of-field image so as to obtain gradient information of pixels of the full depth-of-field image, and generating the feature data based on the gradient information and the preset threshold condition.

4. The method according to claim 3 , wherein the full depth-of-field image generated in step (A1) is composed of pixels having the pixel values of the central pixels thus obtained.

5. The method according to claim 3 , wherein, in sub-step (A2), the gradient operation is performed using an operator selected from the group consisting of the Canny operator, the Sobel operator and the Laplacian operator.

6. The method according to claim 3 , wherein, in sub-step (A2), each of the pixels of the full depth-of-field image having the gradient information with a magnitude greater than the preset threshold condition forms a respective one of the feature elements of the feature data.

7. The method according to claim 1 , wherein step (A) includes the sub-steps of:

(A3) performing, by the image rendering system, geometric transformations upon pixels of the micro-images of the raw light-field image so as to generate an epipolar image; and

(A4) performing, by the image rendering system, a gradient operation upon the epipolar image so as to obtain gradient information of pixels of the epipolar image, and generating the feature data based on the gradient information and the preset threshold condition.

8. The method according to claim 7 , wherein, in sub-step (A4), the gradient operation is performed using an operator selected from the group consisting of the Canny operator, the Sobel operator and the Laplacian operator.

9. The method according to claim 7 , wherein, in sub-step (A4), each of the pixels of the epipolar image having the gradient information with an absolute value greater than the preset threshold condition forms a respective one of the feature elements of the feature data.

10. The method according to claim 1 , wherein, in step (B), each of the pixels of the raw focused image corresponds to a respective one of the micro-images of the raw light-field image, and has a pixel value equal to that of a central pixel of the respective one of the micro-images.

11. The method according to claim 1 , wherein, in step (B), each of the pixels of the raw focused image corresponds to a respective one of the micro-images of the raw light-field image, and has a pixel value equal to an average or a weighted average of pixel values of the respective one of the micro-images.

12. A system for rendering an image from a light-field camera, the light-field camera generating a raw light-field image in response to detecting light rays incident on a focal plane using a photosensor which detects the light rays from a scene, the light rays being incident on an array of microlenses and passed to different locations on the focal plane, the raw light-field image being composed of a plurality of micro-images, said system comprising:

an image pre-processing unit configured to generate feature data, which includes feature elements associated with corresponding position information, associated with the raw light-field image, and obtained based on the raw light-field image and a preset threshold condition, and configured to generate a raw focused image from the raw light-field image;

a set-up unit configured to obtain a virtual focus position that is designated on the raw focused image; and

a focusing processing unit configured to refocus the raw focused image according to the virtual focus position by updating pixel values of pixels of the raw focused image that correspond respectively in position to the feature elements of the feature data, so as to generate a refocused image;

wherein a first portion of pixels of the refocused image that correspond respectively in position to the feature elements of the feature data has pixel values which are determined by performing a refocusing operation upon the raw light-field image according to the virtual focus position; and

wherein a second portion of the pixels of the refocused image that do not correspond in position to the feature elements of the feature data has pixel values which are determined according to pixel values of pixels of the raw focused image that correspond respectively in position to the second portion of the pixels of the refocused image.

13. The system according to claim 12 , wherein the second portion of the pixels of the refocused image has pixel values equal to the pixel values of the pixels of the raw focused image that correspond respectively in position to the second portion of the pixels of the refocused image.

14. The system according to claim 12 , wherein said image pre-processing unit includes:

an image generating module configured to obtain pixel values of central pixels of the micro-images of the raw light-field image, and configured to generate a full depth-of-field image based on the pixel values obtained thereby; and

a detector module configured to perform a gradient operation upon the full depth-of-field image so as to obtain gradient information of pixels of the full depth-of-field image, and configured to generate the feature data based on the gradient information and the preset threshold condition.

15. The system according to claim 14 , wherein the gradient operation is performed by said detector module using an operator selected from the group consisting of the Canny operator, the Sobel operator and the Laplacian operator.

16. The system according to claim 12 , wherein said image pre-processing unit includes:

an image generating module configured to perform geometric transformations upon pixels of the micro-images of the raw light-field image so as to generate an epipolar image; and

a detector module configured to perform a gradient operation upon the epipolar image so as to obtain gradient information of pixels of the epipolar image, and configured to generate the feature data based on the gradient information and the preset threshold condition.

17. The system according to claim 16 , wherein the gradient operation is performed by said detector module using an operator selected from the group consisting of the Canny operator, the Sobel operator and the Laplacian operator.

18. The system according to claim 12 , wherein each of the pixels of the raw focused image corresponds to a respective one of the micro-images of the raw light-field image, and has a pixel value equal to that of a central pixel of the respective one of the micro-images.

19. The system according to claim 12 , wherein each of the pixels of the raw focused image corresponds to a respective one of the micro-images of the raw light-field image, and has a pixel value equal to an average or a weighted average of pixel values of the respective one of the micro-images.