Depth map generation using bokeh detection

1. A method for generating depth maps, the method comprising:

receiving, by one or more computing devices having one or more processors, a set of image frames, each image frame of the set of image frames including pixels;

aligning, by the one or more computing devices, the set of image frames to provide flows of the pixels in the set of image frames;

for each pixel of each image frame of the set of image frames, determining, by the one or more computing devices, a vote value based at least in part on a voting scheme where each pixel of an image frames of the set of image frames votes for a corresponding pixel in another image frame of the set of image frames according to the flows;

identifying, by the one or more computing devices, any region of bokeh effect in each image frame of the set of image frames based at least in part the determined vote values;

adjusting, by the one or more computing devices, the alignment of the set of image frames based at least in part on the identified regions of bokeh effect; and

generating, by the one or more computing devices, a depth map based at least in part on the adjusted alignment.

2. The method of claim 1 , wherein determining the vote value based at least in part on the voting scheme includes using each pixel in a given image frame of the set of image frames vote for at least one corresponding pixel in every other image frame of the set of image frames.

3. The method of claim 1 , wherein identifying regions of bokeh effect includes identifying areas of a given image frame of the set of image frames having higher vote values as compared to other areas of the given image frame.

4. The method of claim 1 , further comprising:

for each identified region of bokeh effect in each given image frame of the set of image frames, propagating back a highest voting value of the identified region of bokeh effect to corresponding pixels for that identified region of bokeh effect in the given image frame in all other image frames of the set of image frames, and

wherein adjusting the alignment is further based on the propagated highest voting values.

5. The method of claim 4 , further comprising:

using the propagated highest voting values to generate a confidence map, and

wherein adjusting the alignment is further based on the confidence map.

6. The method of claim 1 , wherein determining the vote value based at least in part on the voting scheme includes:

pairing each image frame of the set of image frames with each of the other image frames of the set of image frames; and

conducting the voting scheme between each of the paired image frames.

7. The method of claim 1 , wherein identifying any region of bokeh effect includes comparing a voting values for a given pixel of a given image frames of the set of image frames to a threshold to determine whether the given pixel is within a region of bokeh effect.

8. A system for generating depth maps, the system comprising one or more computing devices having one or more processors configured to:

receive a set of image frames, each image frame of the set of image frames including pixels;

align by the one or more computing devices, the set of image frames to provide flows of the pixels in the set of image frames;

for each pixel of each image frame of the set of image frames, determine a vote value based at least in part on a voting scheme where each pixel of an image frames of the set of image frames votes for a corresponding pixel in another image frame of the set of image frames according to the flows;

identify any region of bokeh effect in each image frame of the set of image frames based at least in part the determined vote values;

adjust the alignment of the set of image frames based at least in part on the identified regions of bokeh effect; and

generate a depth map based at least in part on the adjusted alignment.

9. The system of claim 8 , wherein the one or more processors are further configured to determine the vote value based at least in part on the voting scheme by using each pixel in a given image frame of the set of image frames vote for at least one corresponding pixel in every other image frame of the set of image frames.

10. The system of claim 8 , wherein the one or more processors are further configured to identify regions of bokeh effect by identifying areas of a given image frame of the set of image frames having higher vote values as compared to other areas of the given image frame.

11. The system of claim 8 , wherein the one or more processors are further configured to for each identified region of bokeh effect in each given image frame of the set of image frames, propagate back a highest voting value of the identified region of bokeh effect to corresponding pixels for that identified region of bokeh effect in the given image in all other image frames of the set of image frames, and

adjust the alignment further based on the propagated highest voting values.

12. The system of claim 11 , wherein the one or more processors are further configured to:

use the propagated highest voting values to generate a confidence map, and

adjusting the alignment further based on the confidence map.

13. The system of claim 8 , wherein the one or more processors are further configured to determine the vote value based at least in part on the voting scheme by:

pairing each image frame of the set of image frames with each of the other images of the set of image frames; and

conducting the voting scheme between each of the paired images.

14. The system of claim 8 , wherein the one or more processors are further configured to identify any region of bokeh effect by comparing a voting values for a given pixel of a given image frames of the set of image frames to a threshold to determine whether the given pixel is within a region of bokeh effect.

15. A non-transitory, tangible computer-readable recording medium storing instructions, the instructions, when executed by one or more processors, cause the one or more processors to perform a method for generating depth maps, the method comprising:

receiving a set of image frames, each image frame of the set of image frames including pixels;

aligning the set of image frames to provide flows of the pixels in the set of image frames;

for each pixel of each image frame of the set of image frames, determining a vote value based at least in part on a voting scheme where each pixel of an image frames of the set of image frames votes for a corresponding pixel in another image frame of the set of image frames according to the flows;

identifying any region of bokeh effect in each image frame of the set of image frames based at least in part on the determined vote values;

adjusting the alignment of the set of image frames based at least in part on the identified regions of bokeh effect; and

generating a depth map based at least in part on the adjusted alignment.

16. The medium of claim 15 , wherein determining the vote value based at least in part on the voting scheme includes using each pixel in a given image frame of the set of image frames vote for at least one corresponding pixel in every other image frame of the set of image frames.

17. The medium of claim 15 , wherein identifying regions of bokeh effect includes identifying areas of a given image frame of the set of image frames having higher vote values as compared to other areas of the given image frame.

18. The medium of claim 15 , wherein the method further comprises:

for each identified region of bokeh effect in each given image frame of the set of image frames, propagating back a highest voting value of the identified region of bokeh effect to corresponding pixels for that identified region of bokeh effect in the given image frame in all other image frames of the set of image frames, and

wherein adjusting the alignment is further based on the propagated highest voting values.

19. The medium of claim 15 , wherein determining the vote value based at least in part on the voting scheme includes:

pairing each image frame of the set of image frames with each of the other images of the set of image frames; and

conducting the voting scheme between each of the paired images.

20. The medium of claim 15 , wherein identifying any region of bokeh effect includes comparing a voting values for a given pixel of a given image frames of the set of image frames to a threshold to determine whether the given pixel is within a region of bokeh effect.