Road condition deep learning model
The technology relates to using on-board sensor data, off-board information and a deep learning model to classify road wetness and/or to perform a regression analysis on road wetness based on a set of input information. Such information includes on-board and/or off-board signals obtained from one or more sources including on-board perception sensors, other on-board modules, external weather measurement, external weather services, etc. The ground truth includes measurements of water film thickness and/or ice coverage on road surfaces. The ground truth, on-board and off-board signals are used to build the model. The constructed model can be deployed in autonomous vehicles for classifying/regressing the road wetness with on-board and/or off-board signals as the input, without referring to the ground truth. The model can be applied in a variety of ways to enhance autonomous vehicle operation, for instance by altering current driving actions, modifying planned routes or trajectories, activating on-board cleaning systems, etc.
1. A system configured to operate a vehicle in an autonomous driving mode, the system comprising:
a driving system configured to perform driving operations of the vehicle;
a perception system configured to detect objects or conditions in an environment around the vehicle during operation in the autonomous driving mode;
one or more processors operatively coupled to the driving system and the perception system, the one or more processors being configured to:
receive sensor data from the perception system of the vehicle while operating in the autonomous driving mode;
use a stored road condition model to generate information associated with a classification or estimation of road wetness based on the received sensor data; and
cause the driving system to perform a selected driving operation in the autonomous driving mode in response to the generated information.
2. The system of claim 1 , further comprising memory configured to store the road condition model, the memory being operatively coupled to the one or more processors.
3. The system of claim 1 , the model having been trained according to a first set of sensor data training inputs of an environment along a portion of a roadway and a second set of ground truth training inputs.
4. The system of claim 3 , wherein the first set of sensor data training inputs is obtained from one or more on-board sensors of a vehicle operating on the portion of the roadway.
5. The system of claim 3 , wherein the second set of ground truth training inputs comprises off-board information with respect to ground truth data for the portion of the roadway.
6. The system of claim 5 , wherein the ground truth data includes a set of water thickness measurements across one or more areas of the portion of the roadway.
7. The system of claim 3 , wherein the first set of sensor data training inputs includes one or more of lidar returns, still imagery, video imagery, radar returns, audio signals, or output from a vehicle on-board module.
8. The system of claim 3 , wherein the second set of ground truth training inputs includes one or more of weather station information, public weather forecasts, road graph data, crowdsourced information, or observations from a set of vehicles.
9. The system of claim 1 , wherein the selected driving operation includes alteration of a current driving action of the vehicle in the autonomous driving mode.
10. The system of claim 1 , wherein the selected driving operation includes modification of a planned route of the vehicle.
11. The system of claim 1 , wherein the selected driving operation includes modification of a trajectory of the vehicle.
12. The system of claim 1 , wherein the selected driving operation includes activation of an on-board cleaning system.
13. The system of claim 1 , wherein the selected driving operation includes performance of a safety precaution.
14. The system of claim 1 , wherein the one or more processors are further configured to modify a threshold for one or more aspects of the perception system.
15. The system of claim 14 , wherein modification of the threshold for the one or more aspects of the perception system including at least one of modification of a threshold for filtering, a sensor noise level, a sensor field of view adaptation, sensor validation logic, or a pedestrian detector.
16. The system of claim 1 , wherein the one or more processors are further configured to change a model for predicting behavior of other road users based on the generated information.
17. A method comprising:
receiving, by one or more processors, sensor data from one or more sensors of a perception system of a vehicle, the one or more sensors being configured to detect objects or conditions in an environment around the vehicle;
using, by the one or more processors, a stored a road condition model to generate information associated with a classification or estimation of road wetness based on the received sensor data; and
causing, by the one or more processors, a driving system of the vehicle to perform a selected driving operation in an autonomous driving mode in response to the generated information.
18. The method of claim 17 , wherein causing the driving system of the vehicle to perform the selected driving operation in the autonomous driving mode includes at least one of:
triggering one or more safety precautions; or
modifying a threshold for one or more aspects of the perception system.
19. The method of claim 18 , wherein causing the driving system of the vehicle to perform the selected driving operation in the autonomous driving mode includes at least one of:
altering a current driving action of the vehicle in the autonomous driving mode; or
modifying a planned route or trajectory of the vehicle.
20. The method of claim 18 , wherein causing the driving system of the vehicle to perform the selected driving operation in the autonomous driving mode includes activating an on-board cleaning system of the vehicle.