Depth of field adjustment in images based on time of flight depth maps
An image capturing apparatus and a method for depth of filed (DOF) adjustment in images based on time of flight (TOF) depth maps is provided. The image capturing apparatus includes an image sensor and circuitry. The circuitry generates a TOF depth map of a scene that includes a plurality of objects. The TOF depth map includes information associated with distances between the image sensor and surfaces of the plurality of objects. The circuitry divides the TOF depth map into a plurality of regions that corresponds to at least one object of the plurality of objects. The circuitry determines a region of interest from the plurality of regions and adjusts the DOF of the at least one object associated with the determined region of interest. The circuitry further controls the image sensor to capture an image of the scene based on the adjusted DOF.
1. An image capturing apparatus, comprising:
an image sensor;
at least one illuminator; and
circuitry configured to:
control the at least one illuminator to emit first light signals to irradiate a plurality of objects in a scene;
control the image sensor to receive second light signals that correspond to a reflection of the emitted first light signals from surfaces of the plurality of objects in the scene;
calculate a Time of flight (TOF) of the emitted first light signals based on the emitted first light signals and the received second light signals;
calculate distances between the image sensor and the surfaces of the plurality of objects based on the calculated TOF;
generate, based on the calculated distances, a TOF depth map of the scene that includes the plurality of objects, wherein the TOF depth map includes information associated with the distances between the image sensor and the surfaces of the plurality of objects;
divide the generated TOF depth map into a plurality of regions, wherein each of the plurality of regions includes at least one object of the plurality of objects;
determine a region of interest from the plurality of regions;
adjust a Depth of field (DOF) of the at least one object from the plurality of objects, wherein the at least one object corresponds to the region of interest in the generated TOF depth map; and
control the image sensor to capture an image of the scene, wherein the capture of the image is based on the adjusted DOF.
2. The image capturing apparatus according to claim 1 , wherein
the circuitry is further configured to blur a portion of the captured image,
the blurred portion corresponds to a region of the plurality of regions, and
the region is different from the region of interest.
3. The image capturing apparatus according to claim 1 , wherein the region of interest corresponds to a Plane of Focus (POF) of the image sensor.
4. The image capturing apparatus according to claim 1 , further comprising an eye movement detector circuit,
wherein the circuitry is further configured to:
control the eye movement detector circuit to track an eye movement of a user associated with the image capturing apparatus; and
determine the region of interest based on the tracked eye movement of the user.
5. The image capturing apparatus according to claim 1 , further comprising:
at least one lens; and
a lens actuator,
wherein the circuitry is further configured to control, based on the determined region of interest, the lens actuator to adjust a position of the at least one lens with respect to a focal plane of the image sensor.
6. The image capturing apparatus according to claim 5 , further comprising an aperture size controller, wherein
the circuitry is further configured to:
control, based on the determined region of interest, the aperture size controller to set an aperture size of an aperture stop of the image capturing apparatus; and
adjust the DOF of the at least one object based on the set aperture size and the adjusted position of the at least one lens.
7. The image capturing apparatus according to claim 1 , wherein the region of interest corresponds to a center position of the scene.
8. The image capturing apparatus according to claim 1 , wherein the circuitry is further configured to:
receive a user input; and
determine the region of interest from the plurality of regions based on the received user input.
9. The image capturing apparatus according to claim 1 , wherein the circuitry is further configured to:
detect the plurality of objects based on the generated TOF depth map; and
divide the generated TOF depth map into the plurality of regions based on the detection of the plurality of objects.
10. The image capturing apparatus according to claim 1 , wherein the circuitry is further configured to divide the generated TOF depth map into the plurality of regions based on a size of each of the plurality of objects in the generated TOF depth map.
11. A method, comprising:
in an image capturing apparatus that includes an image sensor and at least one illuminator:
controlling the at least one illuminator to emit first light signals to irradiate a plurality of objects in a scene;
controlling the image sensor to receive second light signals that correspond to a reflection of the emitted first light signals from surfaces of the plurality of objects in the scene;
calculating a Time of flight (TOF) of the emitted first light signals based on the emitted first light signals and the received second light signals;
calculating distances between the image sensor and the surfaces of the plurality of objects based on the calculated TOF;
generating, based on the calculated distances, a TOF depth map of the scene that includes the plurality of objects, wherein the TOF depth map includes information associated with the distances between the image sensor and the surfaces of the plurality of objects;
dividing the generated TOF depth map into a plurality of regions, wherein each of the plurality of regions includes at least one object of the plurality of objects;
determining a region of interest from the plurality of regions;
adjusting a Depth of field (DOF) of the at least one object from the plurality of objects, wherein the at least one object corresponds to the region of interest in the generated TOF depth map; and
controlling the image sensor to capture an image of the scene, wherein the capture of the image is based on the adjusted DOF.
12. The method according to claim 11 , further comprising blurring a portion of the captured image, wherein
the blurred portion corresponds to a region of the plurality of regions, and
the region is different from the region of interest.
13. The method according to claim 11 , wherein the region of interest corresponds to a Plane of Focus (POF) of the image sensor.
14. The method according to claim 11 , further comprising:
tracking an eye movement of a user associated with the image capturing apparatus; and
determining the region of interest based on the tracked eye movement of the user.
15. The method according to claim 11 , further comprising controlling, based on the determined region of interest, a lens actuator of the image capturing apparatus to adjust a position of at least one lens of the image capturing apparatus with respect to a focal plane of the image sensor.
16. The method according to claim 15 , further comprising:
controlling, based on the determined region of interest, an aperture size controller to set an aperture size of an aperture stop of the image capturing apparatus; and
adjusting the DOF of the at least one object based on the set aperture size and the adjusted position of the at least one lens.
17. The method according to claim 11 , wherein the region of interest corresponds to a center position of the scene.
18. The method according to claim 11 , further comprising:
receiving a user input; and
determining the region of interest from the plurality of regions based on the received user input.