Methods and systems for learning safe driving paths

1 . A method for determining a safe driving trajectory for an autonomous vehicle in adversity conditions, comprising:

acquiring, by a server computer system including one or more processors, vehicle sensor data for a plurality of driving events, the vehicle sensor data acquired by a plurality of autonomous vehicles, the vehicle sensor data including a first subset of the vehicle sensor data associated with autonomous driving adversity conditions and a second subset of the vehicle sensor data associated with autonomous driving non-adversity conditions, the vehicle sensor data including, for each driving event, corresponding image data depicting surroundings of the vehicle and corresponding inertial measurement data, the inertial measurement data indicating positions of the vehicle, wherein the autonomous driving adversity conditions include conditions that increase a likelihood of failure for an autonomous driving system of an autonomous vehicle in determining a safe driving path trajectory;

acquiring, by the server computer system, for each driving event of the plurality of driving events, corresponding vehicle positioning data indicative of a corresponding trajectory followed by the vehicle, based at least in part on the inertial measurement data;

training, by the server computer system, using at least the first subset of the vehicle sensor data and the vehicle positioning data, one or more machine learning models to predict navigation trajectories during the autonomous driving adversity conditions, training the one or more machine learning models to predict the navigation trajectories further comprising:

extracting first features from the vehicle sensor data;

predicting, using the one or more machine learning models, the navigation trajectories, based on the first features;

comparing the predicted navigation trajectories with the inertial measurement data; and

updating parameters of the one or more machine learning models, based on the comparison; and

training, by the server computer system, using the first subset and the second subset of the vehicle sensor data, the one or more machine learning models to detect the autonomous driving adversity conditions, training the one or more machine learning models to detect the autonomous driving adversity conditions further comprising:

extracting second features from training data;

predicting, by the one or more machine learning models, an indication whether a vehicle environment associated with the training data is an autonomous driving adversity condition, based on the extracted second features;

comparing the predicted indication with a corresponding label of the training data; and

updating the parameters of the one or more machine learning models, based on the comparison.

2 . The method of claim 1 , comprising training, by the computer system, using the vehicle sensor data and the vehicle positioning data, the one or more machine learning models to detect the autonomous driving adversity conditions.

3 . The method of claim 1 , further comprising:

projecting, by the computer system, for each driving event of the plurality of driving events, a graphical representation of the corresponding trajectory followed by the vehicle on the corresponding image data depicting the surroundings of the vehicle; and

using, by the computer system, trajectories followed by the vehicle projected on image data for the plurality of driving events to train the one or more machine models.

4 . The method of claim 1 , wherein the image data includes at least one of images captured by a camera of the vehicle or images captured by a light detection and ranging (LIDAR) system of the vehicle.

5 . The method of claim 1 , wherein the autonomous driving adversity conditions include at least one of:

a road work zone or a construction zone;

obscured or missing road markings; or

one or more obstacles on or alongside a road segment traveled by the vehicle.

6 . The method of claim 1 , wherein the vehicle positioning data includes at least one of:

data indicative of interactions of an individual operating the vehicle with driving control devices of the vehicle; or

data captured by positioning sensors of the vehicle.

7 . The method of claim 1 , wherein at least a subset of the plurality of driving events is associated with an individual operating the vehicle disabling an autonomous driving mode of the vehicle.

8 . The method of claim 1 , wherein the one or more machine learning models include at least one of:

a neural network;

a random forest;

a statistical classifier;

a Naïve Bayes classifier; or

a hierarchical clusterer.

9 . The method of claim 1 , wherein training the one or more machine learning models to predict navigation trajectories includes training the one or more machine learning models to provide control instructions to navigate the vehicle through a predicted trajectory.

10 . A system for determining a safe driving trajectory for an autonomous vehicle in adversity conditions comprising:

at least one processor; and

a non-transitory computer readable medium storing computer instructions, which when executed cause the system to:

acquire vehicle sensor data for a plurality of driving events, the vehicle sensor data including a first subset of the vehicle sensor data associated with autonomous driving adversity conditions and a second subset of the vehicle sensor data associated with autonomous driving non-adversity conditions, the vehicle sensor data including, for each driving event, corresponding image data depicting surroundings of the vehicle and corresponding inertial measurement data, the inertial measurement data indicating positions of the vehicle, wherein the autonomous driving adversity conditions include conditions that increase a likelihood of failure for an autonomous driving system of an autonomous vehicle in determining a safe driving path trajectory;

acquire, for each driving event of the plurality of driving events, corresponding vehicle positioning data indicative of a corresponding trajectory followed by the vehicle, based at least in part on the inertial measurement data;

train, using at least the first subset of the vehicle sensor data and the vehicle positioning data, one or more machine learning models to predict navigation trajectories during the autonomous driving adversity conditions, train the one or more machine learning models to predict the navigation trajectories further comprising:

extract first features from the vehicle sensor data;

predict, using the one or more machine learning models, the navigation trajectories, based on the first features;

compare the predicted navigation trajectories with the inertial measurement data; and

update parameters of the one or more machine learning models, based on the comparison; and

train, using the first subset and the second subset of the vehicle sensor data, the one or more machine learning models to detect the autonomous driving adversity conditions.

11 . The system according to claim 10 , wherein the computer instructions further cause the system to train, using the vehicle sensor data and the vehicle positioning data, the one or more machine learning models to detect the autonomous driving adversity conditions.

12 . The system of claim 10 , wherein the computer instructions further cause the system to:

project, for each driving event of the plurality of driving events, the corresponding trajectory followed by the vehicle on the corresponding image data depicting the surroundings of the vehicle; and

use trajectories followed by the vehicle projected on image data for the plurality of driving events to train the one or more machine learning models.

13 . The system of claim 10 , wherein the image data includes at least one of images captured by a camera of the vehicle or images captured by a light detection and ranging (LIDAR) system of the vehicle.

14 . The system of claim 10 , wherein the autonomous driving adversity conditions include at least one of:

a road work zone or a construction zone;

obscured or missing road markings; or

one or more obstacles on or alongside a road segment traveled by the vehicle.

15 . The system of claim 10 , wherein the vehicle positioning data includes at least one of:

data indicative of interactions of an individual operating the vehicle with driving control devices of the vehicle; or

data captured by positioning sensors of the vehicle.

16 . The system of claim 10 , wherein at least a subset of the plurality of driving events is associated with an individual operating the vehicle disabling an autonomous driving mode of the vehicle.

17 . The system of claim 10 , wherein in training the one or more machine learning models to predict navigation trajectories, the computer instructions cause the system to train the one or more machine learning models to provide control instructions to navigate the vehicle through a predicted trajectory.

18 . A non-transitory computer-readable medium for determining a safe driving trajectory for an autonomous vehicle in adversity conditions, the computer-readable medium comprising computer instructions, the computer instructions when executed by one or more processors cause the one or more processors to:

acquire vehicle sensor data for a plurality of driving events, the vehicle sensor data including a first subset of the vehicle sensor data associated with autonomous driving adversity conditions and a second subset of the vehicle sensor data associated with autonomous driving non-adversity conditions, the vehicle sensor data including, for each driving event, corresponding image data depicting surroundings of the vehicle and corresponding inertial measurement data, the inertial measurement data indicating positions of the vehicle, wherein the autonomous driving adversity conditions include conditions that increase a likelihood of failure for an autonomous driving system of an autonomous vehicle in determining a safe driving path trajectory;

acquire, for each driving event of the plurality of driving events, corresponding vehicle positioning data indicative of a corresponding trajectory followed by the vehicle, based at least in part on the inertial measurement data;

train, using at least the first subset of the vehicle sensor data and the vehicle positioning data, one or more machine learning models to predict navigation trajectories during the autonomous driving adversity conditions, train the one or more machine learning models to predict the navigation trajectories further comprising:

extract features from the vehicle sensor data;

predict, using the one or more machine learning models, the navigation trajectories, based on the extracted features;

compare the predicted navigation trajectories with the inertial measurement data; and

update parameters of the one or more machine learning models, based on the comparison; and

train, using the first subset and the second subset of the vehicle sensor data, the one or more machine learning models to detect the autonomous driving adversity conditions.

19 . The non-transitory computer-readable medium of claim 18 , wherein the one or more machine learning models include at least one first machine learning model and at least one second machine learning model, wherein the at least one first machine learning model is separate from the at least one second machine learning model, and wherein the computer instructions when executed by one or more processors cause the one or more processors to:

train, using at least the first subset of the vehicle sensor data and the vehicle positioning data, the at least one first machine learning model to predict navigation trajectories during the autonomous driving adversity conditions; and

train, using the first subset and the second subset of the vehicle sensor data, the at least one second machine learning model to detect the autonomous driving adversity conditions.

20 . The system of claim 10 , wherein the computer instructions further cause the system to:

train the one or more machine learning models to detect the autonomous driving adversity conditions by:

extracting second features from training data;

predicting, by the one or more machine learning models, an indication whether a vehicle environment associated with the training data is an autonomous driving adversity condition, based on the extracted second features;

comparing the predicted indication with a corresponding label of the training data; and

updating the parameters of the one or more machine learning models, based on the comparison.