Controlling image focus in real-time using gestures and depth sensor data

1. A method of controlling image focus in a digital imaging system, the method comprising:

receiving, by at least one processor of the digital imaging system, a depth-scene image pair generated by a depth-scene image sensor pair included in the digital imaging system, the depth-scene image sensor pair including a depth sensor and a scene sensor, the depth-scene image pair including a depth image and a scene image;

aligning, by the at least one processor, the depth image with the scene image, wherein each pixel in the scene image has a corresponding depth in the aligned depth image;

detecting, by the at least one processor, a user gesture in a scene image stream generated by the scene sensor;

selecting, by the at least one processor, a focus effect to be either a depth of field focus effect or a view camera focus effect, in response to the user gesture; and

applying, by the at least one processor, the selected focus effect to at least a portion of the scene image based on depths in the depth image.

2. The method of claim 1 , wherein the user gesture is a hand gesture.

3. The method of claim 1 , wherein the depth sensor is a time-of-flight (TOF) sensor.

4. The method of claim 3 , wherein detecting the user gesture comprises detecting the user gesture in a depth image stream generated by the TOF sensor.

5. The method of claim 1 , wherein the scene sensor is a red-green-blue (RGB) image sensor.

6. The method of claim 1 , wherein the selected focus effect is the view camera focus effect.

7. The method of claim 1 , wherein the selected focus effect is the depth of field focus effect.

8. The method of claim 1 , further comprising:

displaying the scene image after applying the selected focus effect.

9. A digital imaging system, comprising:

a depth-scene image sensor pair configured to capture depth-scene image pairs of a scene, the depth-scene image sensor pair including a depth sensor and a scene sensor;

at least one processor; and

a non-transitory computer readable storage medium storing a program for execution by the at least one processor, the program including instructions for:

receiving a depth-scene image pair generated by the depth-scene image sensor pair, the depth-scene image pair including a depth image and a scene image;

aligning the depth image with the scene image, wherein each pixel in the scene image has a corresponding depth in the aligned depth image;

detecting a user gesture in a scene image stream generated by the scene sensor;

selecting a focus effect to be either a depth of field focus effect or a view camera focus effect, in response to the user gesture; and

applying the selected focus effect to at least a portion of the scene image based on depths in the depth image.

10. The digital imaging system of claim 9 , wherein the user gesture is a hand gesture.

11. The digital imaging system of claim 9 , wherein the depth sensor is a time-of-flight (TOF) sensor.

12. The digital imaging system of claim 9 , wherein detecting [ [a]] the user gesture comprises detecting the user gesture in a depth image stream generated by the TOF sensor.

13. The digital imaging system of claim 9 , wherein the scene sensor is a red-green-blue (RGB) image sensor.

14. The digital imaging system of claim 9 , wherein the selected focus effect is the view camera focus effect.

15. The digital imaging system of claim 9 , wherein the selected focus effect is the depth of field focus effect.

16. The digital imaging system of claim 9 , wherein the instructions further comprise instructions for:

displaying the scene image after applying the selected focus effect.

17. A non-transitory computer readable storage medium storing a program for execution by at least one processor in a digital imaging system, the program including instructions to:

receive a depth-scene image pair generated by a depth-scene image sensor pair included in the digital imaging system, the depth-scene image sensor pair including a depth sensor and a scene sensor, the depth-scene image pair including a depth image and a scene image;

align the depth image with the scene image wherein each pixel in the scene image has a corresponding depth in the aligned depth image;

detect a user gesture in a scene image stream generated by the scene sensor;

select a focus effect to be either a depth of field focus effect or a view camera focus effect, in response to the user gesture; and

apply the selected focus effect to at least a portion of the scene image based on depths in the depth image.

18. The computer readable storage medium of claim 17 , wherein the user gesture is a hand gesture.

19. The method of claim 7 , wherein applying the selected focus effect comprises:

detecting an object in the scene image;

determining a depth of the object; and

blurring portions of the scene image having depths different than the depth of the object.

20. The method of claim 7 , wherein applying the selected focus effect comprises adjusting a depth of focus based on the user gesture.