Teleassistance data encoding for self-driving vehicles
A self-driving vehicle (SDV) can analyze a live sensor view to autonomously operate acceleration, braking, and steering systems of the SDV along a current route. The SDV can identify an indeterminate object in the live sensor view, and encoding sensor data identifying the indeterminate object for transmission to a backend transport system over one or more networks. The SDV may then receive a resolution response from the backend transport system to resolve the indeterminate object, and cause the SDV to proceed in accordance with the resolution response.
1. A self-driving vehicle (SDV) comprising:
a plurality of communication modules to communicate with a backend transport system;
a sensor system generating sensor data providing a live sensor view of a situational environment of the SDV;
acceleration, braking, and steering systems; and
a control system to execute instructions that cause the control system to:
monitor bandwidth of each of the plurality of communication modules to determine a highest bandwidth communication module;
analyze the live sensor view to autonomously operate the acceleration, braking, and steering systems along a current route;
identify an indeterminate object in the live sensor view by determining that the indeterminate object does not meet a certainty threshold;
in response to identifying the indeterminate object, specify the indeterminate object in a bounding box;
encode sensor data identifying the indeterminate object within the bounding box for transmission over one or more networks by (i) encoding the bounding box at a first bitrate, and (ii) encoding a remainder of the sensor data at a second bitrate;
transmit the encoded sensor data to the backend transport system using the highest bandwidth communication module;
receive a resolution response from the backend transport system to resolve the indeterminate object; and
cause the SDV to proceed in accordance with the resolution response.
2. The SDV of claim 1 , wherein the resolution response classifies the indeterminate object.
3. The SDV of claim 1 , wherein the sensor system comprises at least a LIDAR system and a camera system that combine to generate the live sensor view.
4. The SDV of claim 3 , wherein the encoded sensor data comprises image data from at least one camera sensor of the camera system.
5. The SDV of claim 1 , wherein the executed instructions further cause the control system to:
detect, in the live sensor view, objects of interest along the current route; and
attempt to classify each of the objects of interest in accordance with the certainty threshold.
6. The SDV of claim 1 , wherein the resolution response is determined by one or more human teleassistance operators.
7. The SDV of claim 1 , wherein the executed instructions further cause the control system to:
duplicate transmission of the of the encoded sensor data by transmitting the encoded sensor data to the backend transport system using a second communication module of the plurality of communication modules.
8. A non-transitory computer readable medium storing instructions that, when executed by a control system of a self-driving vehicle (SDV), cause the control system to:
monitor bandwidth of each of a plurality of communication modules of the SDV to determine a highest bandwidth communication module;
analyze a live sensor view of the SDV to autonomously operate acceleration, braking, and steering systems of the SDV along a current route;
identify an indeterminate object in the live sensor view by determining that the indeterminate object does not meet a certainty threshold;
in response to identifying the indeterminate object, specify the indeterminate object in a bounding box;
encode sensor data identifying the indeterminate object within the bounding box for transmission over one or more networks by (i) encoding the bounding box at a first bitrate, and (ii) encoding a remainder of the sensor data at a second bitrate;
transmit the encoded sensor data to a backend transport system over the one or more networks using the highest bandwidth communication module;
receive, over the one or more networks, a resolution response from the backend transport system to resolve the indeterminate object; and
cause the SDV to proceed in accordance with the resolution response.
9. The non-transitory computer readable medium of claim 8 , wherein the executed instructions further cause the control system to:
duplicate transmission of the of the encoded sensor data by transmitting the encoded sensor data to the backend transport system using a second communication module of the plurality of communication modules.
10. The non-transitory computer readable medium of claim 9 , wherein the resolution response classifies the indeterminate object.
11. The non-transitory computer readable medium of claim 9 , wherein the SDV comprises at least a LIDAR system and a camera system that combine to generate the live sensor view.
12. The non-transitory computer readable medium of claim 11 , wherein the encoded sensor data comprises image data from at least one camera sensor of the camera system.
13. The non-transitory computer readable medium of claim 9 , wherein the executed instructions further cause the control system to:
detect, in the live sensor view, objects of interest along the current route; and
attempt to classify each of the objects of interest in accordance with the certainty threshold.
14. The non-transitory computer readable medium of claim 9 , wherein the resolution response is determined by human teleassistance operators.
15. A computer-implemented method of operating a self-driving vehicle (SDV), the method being performed by one or more processors and comprising:
monitoring bandwidth of each of a plurality of communication modules of the SDV to determine a highest bandwidth communication module;
analyzing a live sensor view of the SDV to autonomously operate acceleration, braking, and steering systems of the SDV along a current route;
identifying an indeterminate object in the live sensor view by determining that the indeterminate object does not meet a certainty threshold;
in response to identifying the indeterminate object, specify the indeterminate object in a bounding box;
encoding sensor data identifying the indeterminate object within the bounding box for transmission over one or more networks by (i) encoding the bounding box at a first bitrate, and (ii) encoding a remainder of the sensor data at a second bitrate;
transmitting the encoded sensor data to a backend transport system over the one or more networks using the highest bandwidth communication module;
receiving, over the one or more networks, a resolution response from the backend transport system to resolve the indeterminate object; and
causing the SDV to proceed in accordance with the resolution response.
16. The method of claim 15 , wherein the resolution response classifies the indeterminate object.
17. The method of claim 15 , wherein the SDV comprises at least a LIDAR system and a camera system that combine to generate the live sensor view.
18. The method of claim 17 , wherein the encoded sensor data comprises image data from at least one camera sensor of the camera system.
19. The method of claim 15 , further comprising:
detecting, in the live sensor view, objects of interest along the current route; and
attempting to classify each of the objects of interest in accordance with a certainty threshold.
20. The method of claim 15 , further comprising:
duplicating transmission of the of the encoded sensor data by transmitting the encoded sensor data to the backend transport system using a second communication module of the plurality of communication modules.