Assisted perception for autonomous vehicles
Disclosed herein are systems and methods for providing supplemental identification abilities to an autonomous vehicle system. The sensor unit of the vehicle may be configured to receive data indicating an environment of the vehicle, while the control system may be configured to operate the vehicle. The vehicle may also include a processing unit configured to analyze the data indicating the environment to determine at least one object having a detection confidence below a threshold. Based on the at least one object having a detection confidence below a threshold, the processor may communicate at least a subset of the data indicating the environment for further processing. The vehicle is also configured to receive an indication of an object confirmation of the subset of the data. Based on the object confirmation of the subset of the data, the processor may alter the control of the vehicle by the control system.
1. A method comprising:
receiving, at a first computing device, sensor data indicative of a surrounding environment from a sensor coupled to a vehicle;
based on the sensor data, identifying an object in the surrounding environment;
determining a confidence level for an identification of the object;
based on the confidence level for the identification of the object being below a confidence threshold, controlling a movement of the vehicle to maintain a safe distance from the object;
providing, by the first computing device to a second computing device, an object inquiry that includes one or more images showing the object in a bounding box and the identification of the object as determined by the first computing device, wherein the second computing device is configured to display the one or more images showing the object in the bounding box and the identification of the object;
receiving, at the first computing device, a response to the object inquiry from the second computing device; and
controlling the vehicle according to a control strategy generated based on the response to the object inquiry.
2. The method of claim 1 , wherein the first computing device is positioned onboard the vehicle, and
wherein receiving sensor data indicative of the surrounding environment from the sensor coupled to the vehicle comprises:
receiving first sensor data indicative of the surrounding environment from a camera coupled to the vehicle; and
receiving second sensor data indicative of the surrounding environment from a radar system coupled to the vehicle.
3. The method of claim 2 , wherein identifying the object in the surrounding environment comprises:
identifying the object based on a combination of the first sensor data and the second sensor data.
4. The method of claim 1 , wherein determining the confidence level for the identification of the object comprises:
comparing the identification of the object to a plurality of identified objects stored in memory; and
determining the confidence level based on comparing the identification of the object to the plurality of identified objects.
5. The method of claim 1 , wherein providing the object inquiry to the second computing device further comprises:
displaying an image to a passenger located in the vehicle.
6. The method of claim 1 , wherein identifying the object in the surrounding environment comprises:
determining that the object is located in a path of the vehicle;
wherein receiving the response to the inquiry from the second computing device comprises:
receiving an indication that the object is an erroneous detection; and
wherein controlling the vehicle based on the response to the object inquiry further comprises:
causing the vehicle to continue to navigate the path of the vehicle based on receiving the indication that the object is the erroneous detection.
7. The method of claim 1 , wherein identifying the object in the surrounding environment comprises:
identifying the object as a person directing traffic;
wherein determining the confidence level for the identification of the object comprises:
determining that the confidence level for the identification of the person directing traffic is below the confidence threshold; and
wherein providing the object inquiry to the second computing device comprises:
providing video of the person directing traffic to the second computing device.
8. The method of claim 1 , wherein identifying the object in the surrounding environment comprises:
detecting a particular sound using a microphone coupled to the vehicle; and
identifying the particular sound as a siren.
9. The method of claim 1 , wherein identifying the object in the surrounding environment comprises:
identifying the object as a motorcycle operating nearby the vehicle.
10. The method of claim 1 , further comprising:
based on the confidence level for the identification of the object being above the confidence threshold, determining an alternative path for the vehicle that avoids the object; and
controlling the vehicle based on the alternative path.
11. The method of claim 1 , wherein the second computing device is a remote computing device positioned away from the vehicle.
12. A system comprising:
a sensor coupled to a vehicle;
a first computing device, wherein the first computing device is configured to:
receive sensor data indicative of a surrounding environment from a sensor coupled to a vehicle;
based on the sensor data, identify an object in the surrounding environment;
determine a confidence level for an identification of the object;
based on the confidence level for the identification of the object being below a confidence threshold, control a movement of the vehicle to maintain a safe distance from the object;
provide an object inquiry to a second computing device, wherein the object inquiry includes one or more images showing the object in a bounding box and the identification of the object as determined by the first computing device, and wherein the second computing device is configured to display the one or more images showing the object in the bound box and the identification of the object;
receive a response to the object inquiry from the second computing device; and
control the vehicle according to a control strategy generated based on the response to the object inquiry.
13. The system of claim 12 , wherein the first computing device is further configured to:
identify the object in the surrounding environment based on a context associated with the surrounding environment.
14. The system of claim 12 , wherein the first computing device is further configured to:
provide the sensor data with the object inquiry.
15. The system of claim 12 , wherein the first computing device is coupled to the vehicle and the second computing device is positioned remote from the vehicle.
16. A non-transitory computer-readable medium having stored thereon program instructions that, if executed by a first computing device, causes the first computing device to perform operations comprises:
receiving sensor data indicative of a surrounding environment from a sensor coupled to a vehicle;
based on the sensor data, identifying an object in the surrounding environment;
determining a confidence level for an identification of the object;
based on the confidence level for the identification of the object being below a confidence threshold, controlling a movement of the vehicle to maintain a safe distance from the object;
providing an object inquiry to a second computing device, wherein the object inquiry includes one or more images showing the object in a bounding box and the identification of the object as determined by the first computing device, and wherein the second computing device is configured to display the one or more images showing the object in the bounding box and the identification of the object;
receiving a response to the object inquiry from the second computing device; and
controlling the vehicle according to a control strategy generated based on the response to the object inquiry.
17. The non-transitory computer-readable medium of claim 16 , wherein
wherein receiving sensor data indicative of the surrounding environment from the sensor coupled to the vehicle comprises:
receiving first sensor data indicative of the surrounding environment from a camera coupled to the vehicle; and
receiving second sensor data indicative of the surrounding environment from a radar system coupled to the vehicle.
18. The non-transitory computer-readable medium of claim 17 , wherein identifying the object in the surrounding environment comprises:
identifying the object based on a combination of the first sensor data and the second sensor data.
19. The non-transitory computer-readable medium of claim 16 , wherein determining the confidence level for the identification of the object comprises:
comparing the identification of the object to a plurality of identified objects stored in memory; and
determining the confidence level based on comparing the identification of the object to the plurality of identified objects.
20. The non-transitory computer-readable medium of claim 16 , wherein the second computing device is a remote computing device positioned away from the vehicle.