Method and device for generating all-in-focus image

1 . A method of generating an all-in-focus image, the method comprising:

obtaining a scene image and a phase detection image based on capturing a scene using a phase detection image sensor;

determining a blur kernel based on the phase detection image and a mapping relationship, the mapping relationship indicating a depth and a blur kernel distribution;

and generating the all-in-focus image based on deblurring the scene image using the blur kernel,

wherein the phase detection image includes

a depth image, the depth image including a plurality of depth values corresponding to a plurality of depth pixels, each depth value of the plurality of depth values corresponding to a different depth pixel of the plurality of depth pixels,

and a confidence image, the confidence image including a plurality of confidence values, each confidence value of the plurality of confidence values corresponds to a different depth value of the plurality of depth values and indicates a credibility of the different depth value,

and wherein the determining the blur kernel based on the phase detection image and the mapping relationship includes

refining the depth image to a refined depth image, the refining including adjusting a separate depth value of the plurality of depth values to a separate refined depth value based on a corresponding confidence value of the plurality of confidence values that corresponds to the separate depth value being lower than a threshold value,

determining the blur kernel based on applying the refined depth image to the mapping relationship,

wherein the all-in-focus image is generated based on only one frame,

and wherein the separate refined depth value is equal to the separate depth value in response to the corresponding confidence value being greater than the threshold value.

2 . The method of claim 1 , wherein

the scene image is obtained by pixels for sensing light intensity of the phase detection image sensor, and

the phase detection image is obtained by phase detection pixels of the phase detection image sensor.

3 . The method of claim 1 , wherein the determining the blur kernel based on applying the refined depth image to the mapping relationship comprises:

generating an upscaled depth image having a resolution of the scene image based on upscaling the refined depth image; and

determining the blur kernel based on the upscaled depth image having the resolution of the scene image and the mapping relationship.

4 . The method of claim 1 , wherein the generating the all-in-focus image comprises:

deblurring the scene image based on applying the blur kernel to a blur kernel convolution model to generate the all-in-focus image.

5 . The method of claim 1 , wherein the mapping relationship is established based on:

obtaining blur kernel images based on images of a point source collected by the phase detection image sensor at different depths; and

establishing the mapping relationship indicating the depth and the blur kernel distribution based on interpolating the blur kernel images or fitting the blur kernel images.

6 . The method of claim 1 , wherein the separate refined depth value is a function of the corresponding confidence value.

7 . A device for generating an all-in-focus image, the device comprising:

a memory storing executable instructions;

and a processor that is configured to execute the executable instructions to cause the processor to

obtain a scene image and a phase detection image based on capturing a scene using a phase detection image sensor,

determine a blur kernel based on the phase detection image and a mapping relationship, the mapping relationship indicating a depth and a blur kernel distribution,

and generate the all-in-focus image based on deblurring the scene image using the blur kernel,

wherein the phase detection image includes

a depth image, the depth image including a plurality of depth values corresponding to a plurality of depth pixels, each depth value of the plurality of depth values corresponding to a different depth pixel of the plurality of depth pixels,

and a confidence image, the confidence image including a plurality of confidence values, each confidence value of the plurality of confidence values corresponds to a different depth value of the plurality of depth values and indicates a credibility of the different depth value,

wherein the processor is configured to determine the blur kernel based on the phase detection image and the mapping relationship, including

refining the depth image to a refined depth image, the refining including adjusting a separate depth value of the plurality of depth values to a separate refined depth value based on a corresponding confidence value of the plurality of confidence values that corresponds to the separate depth value being lower than a threshold value,

and determining the blur kernel based on applying the refined depth image to the mapping relationship,

wherein the all-in-focus image is generated based on only one frame,

and wherein the separate refined depth value is equal to the separate depth value in response to the corresponding confidence value being greater than the threshold value.

8 . The device of claim 7 , wherein

the scene image is obtained by pixels for sensing light intensity of the phase detection image sensor, and

the phase detection image is obtained by phase detection pixels of the phase detection image sensor.

9 . The device of claim 7 , wherein the processor is further configured to execute the executable instructions to

generate an upscaled depth image having a resolution of the scene image based on upscaling the refined depth image; and

determine the blur kernel based on the upscaled depth image having the resolution of the scene image and the mapping relationship.

10 . The device of claim 7 , wherein the processor is further configured to execute the executable instructions to

deblur the scene image based on applying the blur kernel to a blur kernel convolution model to generate the all-in-focus image.

11 . The device of claim 7 , wherein the mapping relationship is established based on:

obtaining blur kernel images based on images of a point source collected by the phase detection image sensor at different depths; and

establishing the mapping relationship indicating the depth and the blur kernel distribution based on interpolating the blur kernel images or fitting the blur kernel images.

12 . The device of claim 7 , wherein the separate refined depth value is a function of the corresponding confidence value.