Systems and methods for depth map sampling
An electronic imaging device and method for image capture are described. The imaging device includes a camera configured to obtain image information of a scene and that may be focused on a region of interest in the scene. The imaging device also includes a LIDAR unit configured to obtain depth information of at least a portion of the scene at specified scan locations of the scene. The imaging device is configured to detect an object in the scene and provides specified scan locations to the LIDAR unit. The camera is configured to capture an image with an adjusted focus based on depth information, obtained by the LIDAR unit, associated with the detected object.
1 . An imaging device, comprising:
a depth sensor configured to obtain depth information of at least a portion of a scene; and
a processor configured to:
obtain first depth information obtained from the depth sensor based on a first scan iteration of the depth sensor;
generate a segmentation of image information of the scene;
detect an object in the scene using the image information of the scene based on feedback information including the first depth information based on the first scan iteration of the depth sensor;
provide one or more instructions to the depth sensor to obtain second depth information at a plurality of specified scan locations for at least a second scan iteration of the depth sensor, wherein the plurality of specified scan locations are characterized by a first scan point density for a first portion of the scene corresponding to a first segment associated with the object and a second scan point density for a second portion of the scene corresponding to a second segment, wherein the first scan point density is based on a comparison of image information of neighboring image segments within the object and a comparison of first depth information of the neighboring image segments within the object; and
output the second depth information as feedback for detection of at least one of the object or an additional object in the scene using additional image information of the scene.
2 . The imaging device of claim 1 , wherein the depth sensor is a light-detection and ranging (LIDAR) unit configured to capture LIDAR data.
3 . The imaging device of claim 1 , wherein the processor is configured to:
generate a first image segmentation of the image information associated with the object; and
generate a second image segmentation of the image information associated with a portion of the scene not associated with the object.
4 . The imaging device of claim 3 , wherein one or more scan locations of the first portion of the scene are based on the first image segmentation, and wherein one or more scan locations of the second portion of the scene are based on the second image segmentation.
5 . The imaging device of claim 1 , wherein the first scan point density is greater than the second scan point density.
6 . The imaging device of claim 1 , wherein the object is a person or a face of the person.
7 . The imaging device of claim 6 , wherein the processor is configured:
detect the face using facial recognition.
8 . The imaging device of claim 1 , further comprising a user interface configured to receive user input indicating a selection of the object, wherein the processor is configured to detect the object based on the user input indicating the selection of the object.
9 . The imaging device of claim 1 , wherein the processor is configured to:
generate a depth map for at least a portion of the scene based on the second depth information.
10 . The imaging device of claim 9 , wherein the processor is configured to:
generate a three-dimensional model of the object based on the depth map.
11 . The imaging device of claim 9 , wherein the processor is configured to:
generate an image segmentation of the image information associated with the object; and
generate a refined depth map based on the image segmentation.
12 . The imaging device of claim 1 , wherein the processor is configured to:
determine, based on the second depth information, a focus amount for a camera configured to capture the image information.
13 . The imaging device of claim 1 , wherein the processor is configured to:
detect one or more objects in the scene based on the second depth information.
14 . The imaging device of claim 13 , wherein the processor is configured to:
output an indication of the detected one or more objects in the scene for controlling a vehicle.
15 . A method for image capture, the method comprising:
obtaining first depth information obtained from a depth sensor based on a first scan iteration of the depth sensor;
generating a segmentation of image information of a scene;
detecting an object in the scene using the image information of the scene based on feedback information including the first depth information based on the first scan iteration of the depth sensor;
outputting one or more instructions to the depth sensor to obtain second depth information at a plurality of specified scan locations for at least a second scan iteration of the depth sensor, wherein the plurality of specified scan locations are characterized by a first scan point density for a first portion of the scene corresponding to a first segment associated with the object and a second scan point density for a second portion of the scene corresponding to a second segment, wherein the first scan point density is based on a comparison of image information of neighboring image segments within the object and a comparison of first depth information of the neighboring image segments within the object; and
outputting the second depth information as feedback for detection of at least one of the object or an additional object in the scene using additional image information of the scene.
16 . The method of claim 15 , wherein the depth sensor is a light-detection and ranging (LIDAR) unit configured to capture LIDAR data.
17 . The method of claim 15 , further comprising:
generating a first image segmentation of the image information associated with the object; and
generating a second image segmentation of the image information associated with a portion of the scene not associated with the object.
18 . The method of claim 17 , wherein one or more scan locations of the first portion of the scene are based on the first image segmentation, and wherein one or more scan locations of the second portion of the scene are based on the second image segmentation.
19 . The method of claim 15 , wherein the first scan point density is greater than the second scan point density.
20 . The method of claim 15 , wherein the object is a person or a face of the person.
21 . The method of claim 20 , further comprising:
detecting the face using facial recognition.
22 . The method of claim 15 , further comprising:
receiving, via a user interface, user input indicating a selection of the object; and
detecting the object based on the user input indicating the selection of the object.
23 . The method of claim 15 , further comprising:
generating a depth map for at least a portion of the scene based on the second depth information.
24 . The method of claim 23 , further comprising:
generating a three-dimensional model of the object based on the depth map.
25 . The method of claim 23 , further comprising:
generating an image segmentation of the image information associated with the object; and
generating a refined depth map based on the image segmentation.
26 . The method of claim 15 , further comprising:
determining, based on the second depth information, a focus amount for a camera configured to capture the image information.
27 . The method of claim 15 , further comprising:
detecting one or more objects in the scene based on the second depth information.
28 . The method of claim 27 , further comprising:
outputting an indication of the detected one or more objects in the scene for controlling a vehicle.