Automatic robotically steered sensor for targeted high performance perception and vehicle control
Disclosed are methods, systems, and non-transitory computer readable media that control an autonomous vehicle via at least two sensors. One aspect includes capturing an image of a scene ahead of the vehicle with a first sensor, identifying an object in the scene at a confidence level based on the image, determining the confidence level of the identifying is below a threshold, in response to the confidence level being below the threshold, directing a second sensor having a field of view smaller than the first sensor to generate a second image including a location of the identified object, further identifying the object in the scene based on the second image, controlling the vehicle based on the further identification of the object.
1 . A method of controlling a vehicle, the method comprising:
capturing a first image with a first sensor, wherein the first image includes an object;
based on an attempt to identify the object in the first image, determining a capture location of the object for obtaining a second image of the object by a second sensor, wherein the second sensor is configured with an active illumination device that is synchronized to illuminate when the second sensor captures the second image;
directing the second sensor to capture the second image of the object at the capture location of the object, wherein the active illumination device illuminates a field of view corresponding to the capture location of the object;
identifying the object based on the second image; and
controlling the vehicle based on identifying the object.
2 . The method of claim 1 , wherein directing the second sensor to capture the second image comprises determining an orientation of the vehicle relative to the capture location.
3 . The method of claim 1 , wherein directing the second sensor to capture the second image comprises:
determining a speed associated with the vehicle;
determining a time to position the second sensor from a first position to a second position, wherein the second position is associated with the capture location; and
based on the speed and the time to position the second sensor, directing the second sensor to the second position to capture the second image.
4 . The method of claim 3 , further comprising:
accessing map data associated with a surrounding environment of the vehicle; and
based on the map data, the speed, and the time to position the second sensor, determining a capture location.
5 . The method of claim 1 , wherein the second sensor includes a different field of view than the first sensor.
6 . The method of claim 1 , wherein the second sensor includes a higher pixel density than the first sensor.
7 . The method of claim 1 , wherein the capture location is determined in three dimensional space.
8 . The method of claim 1 , wherein the active illumination device is configured with a field of view that corresponds to a field of view of the second sensor.
9 . A vehicle comprising:
vehicle controls for controlling the vehicle;
a first sensor and a second sensor, wherein the second sensor is configured with an active illumination device;
one or more processors;
a memory storing instructions that are executable by the one or more processors to cause the one or more processors to:
capture a first image with the first sensor, wherein the first image includes an object;
based on an attempt to identify the object in the first image, determine a capture location of the object for obtaining a second image of the object by the second sensor, wherein the active illumination device is synchronized to illuminate when the second sensor captures the second image;
direct the second sensor to capture the second image of the object at the capture location of the object, wherein the active illumination device illuminates a field of view corresponding to the capture location of the object;
identify the object based on the second image; and
control the vehicle based on identifying the object.
10 . The vehicle of claim 9 , wherein directing the second sensor to capture the second image comprises determining an orientation of the vehicle relative to the capture location.
11 . The vehicle of claim 9 , wherein directing the second sensor to capture the second image comprises:
determining a speed associated with the vehicle;
determining a time to position the second sensor from a first position to a second position, wherein the second position is associated with the capture location; and
based on the speed and the time to position the second sensor, directing the second sensor to the second position to capture the second image.
12 . The vehicle of claim 11 , wherein the one or more processors:
access map data associated with a surrounding environment of the vehicle; and
based on the map data, the speed, and the time to position the second sensor, determine a capture location.
13 . The vehicle of claim 9 , wherein the second sensor includes a different field of view than the first sensor.
14 . The vehicle of claim 9 , wherein the second sensor includes a higher pixel density than the first sensor.
15 . The vehicle of claim 9 , wherein the capture location is determined in three dimensional space.
16 . The vehicle of claim 9 , wherein the active illumination device is configured with a field of view that corresponds to a field of view of the second sensor.
17 . A computing system for controlling a vehicle comprising:
one or more processors;
one or more non-transitory computer-readable media that store instructions executable by the one or more processors to cause the one or more processor to perform operations, the operations comprising:
capturing a first image with a first sensor, wherein the first image includes an object;
based on an attempt to identify the object in the first image, determining a capture location of the object for obtaining a second image of the object by a second sensor, wherein the second sensor is configured with an active illumination device that is synchronized to illuminate when the second sensor captures the second image;
directing the second sensor to capture the second image of the object at the capture location of the object, wherein the active illumination device illuminates a field of view corresponding to the capture location of the object;
identifying the object based on the second image; and
controlling the vehicle based on identifying the object.
18 . The computing system of claim 17 , wherein directing the second sensor to capture the second image comprises determining an orientation of the vehicle relative to the capture location.
19 . The computing system of claim 18 , wherein directing the second sensor to capture the second image comprises:
determining a speed associated with the vehicle;
determining a time to position the second sensor from a first position to a second position, wherein the second position is associated with the capture location; and
based on the speed and the time to position the second sensor, directing the second sensor to the second position to capture the second image.
20 . The computing system of claim 19 , wherein the operations further comprise:
accessing map data associated with a surrounding environment of the vehicle; and
based on the map data, the speed, and the time to position the second sensor, determining a capture location.