Multi-channel sensor simulation for autonomous control systems
An autonomous control system combines sensor data from multiple sensors to simulate sensor data from high-capacity sensors. The sensor data contains information related to physical environments surrounding vehicles for autonomous guidance. For example, the sensor data may be in the form of images that visually capture scenes of the surrounding environment, geo-location of the vehicles, and the like. The autonomous control system simulates high-capacity sensor data of the physical environment from replacement sensors that may each have lower capacity than high-capacity sensors. The high-capacity sensor data may be simulated via one or more neural network models. The autonomous control system performs various detection and control algorithms on the simulated sensor data to guide the vehicle autonomously.
1 . A method of controlling a vehicle with objects identified in an environment with simulated sensor data, comprising:
obtaining sensor data from a plurality of low-capacity sensors positioned on a vehicle in a real-world environment, wherein the low-capacity sensors are passive image sensors;
applying a machine-learning model to a synthesized input of the obtained sensor data, wherein the machine-learning model determines simulated information associated with high-capacity sensors; and
adjusting a control parameter of the vehicle based on one or more objects identified, at least in part, using the simulated information.
2 . The method of claim 1 , wherein the simulated information is associated with a LIDAR sensor.
3 . The method of claim 1 , wherein the simulated information simulates measurements with respect to use of a LIDAR sensor in the real-world environment.
4 . The method of claim 1 , wherein the simulated information indicates, at least, depth measurements associated with the identified objects.
5 . The method of claim 1 , wherein the machine-learning model includes a first portion configured to receive sensor data generated by a first subset of the low-capacity sensors and generate a first output, a second portion configured to receive sensor data generated by a second subset of the low-capacity sensors and generate a second output, wherein the first subset and the second subset are image sensors with different resolution and field-of-view, and wherein the simulated information is generated by synthesizing the first output and the second output.
6 . The method of claim 1 , wherein the low-capacity sensors include at least two sensors having differing sensing characteristics, and wherein the simulated information is generated based on the differing sensing characteristics.
7 . The method of claim 1 , wherein the machine-learning model comprises output channels which output confidence scores associated with the simulated information, and wherein the confidence scores are usable by a control system of the vehicle to avoid a portion of the real-world environment or to reduce a speed of the vehicle.
8 . The method of claim 1 , wherein the vehicle does not have high-capacity sensors positioned on the vehicle.
9 . Non-transitory computer-readable media storing instructions for execution on a processor, the instructions when executed by the processor cause the processor to:
obtain sensor data from a plurality of low-capacity sensors positioned on a vehicle in a real-world environment, wherein the low-capacity sensors are passive image sensors;
apply a machine-learning model to a synthesized input of the obtained sensor data, wherein the machine-learning model determines simulated information associated with high-capacity sensors; and
adjust a control parameter of the vehicle based on one or more objects identified, at least in part, using the simulated information.
10 . The computer-readable media of claim 9 , wherein the simulated information is associated with a LIDAR sensor.
11 . The computer-readable media of claim 9 , wherein the simulated information simulates measurements with respect to use of a LIDAR sensor in the real-world environment.
12 . The computer-readable media of claim 9 , wherein the simulated information indicates, at least, depth measurements associated with the identified objects.
13 . The computer-readable media of claim 9 , wherein the machine-learning model includes a first portion configured to receive sensor data generated by a first subset of the low-capacity sensors and generate a first output, a second portion configured to receive sensor data generated by a second subset of the low-capacity sensors and generate a second output, wherein the first subset and the second subset are image sensors with different resolution and field-of-view, and wherein the simulated information is generated by synthesizing the first output and the second output.
14 . The computer-readable media of claim 9 , wherein the machine-learning model comprises output channels which output confidence scores associated with the simulated information, and wherein the confidence scores are usable by a control system of the vehicle to avoid a portion of the real-world environment or to reduce a speed of the vehicle.
15 . A system comprising one or more processors and non- transitory computer storage media storing instructions that when executed by the one or more processors, cause the one or more processors to:
obtain sensor data from a plurality of low-capacity sensors positioned on a vehicle in a real-world environment, wherein the low-capacity sensors are passive image sensors;
apply a machine-learning model to a synthesized input of the obtained sensor data, wherein the machine-learning model determines simulated information associated with high-capacity sensors; and
adjust a control parameter of the vehicle based on one or more objects identified, at least in part, using the simulated information.
16 . The system of claim 15 , wherein the simulated information is associated with a LIDAR sensor.
17 . The system of claim 15 , wherein the simulated information indicates, at least, depth measurements associated with the identified objects.
18 . The system of claim 15 , wherein the machine-learning model comprises output channels which output confidence scores associated with the simulated information, and wherein the confidence scores are usable by a control system of the vehicle to avoid a portion of the real-world environment or to reduce a speed of the vehicle.