Modifying autonomous vehicle behavior prediction based on behavior prediction errors
A system includes a memory device and a processing device, operatively coupled to the memory device, to identify one or more erroneous predictions of behavior for one or more objects in an environment of an autonomous vehicle traveling along a planned trajectory, and initiate, based on the one or more erroneous predictions, one or more operations to adjust the planned trajectory of the autonomous vehicle. Each erroneous prediction of the one or more erroneous predictions is determined based on a comparison between a corresponding observed spatial overlap between the autonomous vehicle and a corresponding object, and a corresponding predicted spatial overlap between the autonomous vehicle and the corresponding object.
1 . A system comprising:
a memory device storing instructions; and
a processing device, operatively coupled to the memory device, the processing device executing the instructions that cause the processing device to:
identify one or more erroneous predictions of behavior for one or more objects in an environment of an autonomous vehicle traveling along a planned trajectory, wherein to identify an erroneous prediction of the one or more erroneous predictions, the processing device is further to:
identify a corresponding observed spatial overlap between the autonomous vehicle and a corresponding object;
identify a corresponding predicted spatial overlap between the autonomous vehicle and the corresponding object; and
determine a mismatch between the corresponding observed spatial overlap and the corresponding predicted spatial overlap; and
initiate, based on the one or more erroneous predictions, one or more operations to adjust the planned trajectory of the autonomous vehicle.
2 . The system of claim 1 , wherein, to identify the one or more erroneous predictions, the processing device is further to:
obtain a set of predicted behavior data indicative of one or more predicted object behaviors for the one or more objects, and a set of observed behavior data indicative of one or more observed object behaviors of the one or more objects;
determine, based on a comparison of the set of predicted behavior data and the set of observed behavior data, whether a set of erroneous predictions exists within the set of predicted behavior data;
obtain the set of erroneous predictions in response to determining that the set of erroneous predictions exists within the set of predicted behavior data; and
generate the one or more erroneous predictions as a subset of the set of erroneous predictions.
3 . The system of claim 2 , wherein the one or more predicted object behaviors indicate one of: that no spatial overlap is predicted between the autonomous vehicle and the corresponding object, that the autonomous vehicle arrives at a predicted spatial overlap region before the corresponding object, that the autonomous vehicle arrives at the predicted spatial overlap region after the corresponding object, or that the autonomous vehicle and the corresponding object arrive at the predicted spatial overlap region at a substantially same time.
4 . The system of claim 2 , wherein the one or more observed object behaviors indicate one of: that no spatial overlap is observed between the autonomous vehicle and the corresponding object, that the autonomous vehicle arrives at an observed spatial overlap region before the corresponding object, that the autonomous vehicle arrives at the observed spatial overlap region after the corresponding object, or that the autonomous vehicle and the corresponding object arrive at the observed spatial overlap region at a substantially same time.
5 . The system of claim 1 , wherein:
the processing device is further to determine whether a first predicted spatial overlap state associated with a first erroneous prediction of behavior of a first object does not match a first observed spatial overlap state associated with the first erroneous prediction, wherein the first erroneous prediction is associated with a first error relevancy indicator of a first relevancy; and
to initiate the one or more operations to adjust the planned trajectory of the autonomous vehicle, the processing device is further to, upon determining that the first predicted spatial overlap state matches the first observed spatial overlap state, associate a second erroneous prediction of behavior of a second object with a second error relevancy indicator of a second relevancy, the first relevancy being greater than the second relevancy, and the first error relevancy indicator representing a higher priority of the first erroneous prediction with respect to the second erroneous prediction within the one or more erroneous predictions.
6 . The system of claim 5 , wherein:
the processing device is further to determine whether a third predicted spatial overlap state associated with a third erroneous prediction of behavior of a third object does not match a third observed spatial overlap state, and a first time-to-overlap with respect to the first observed spatial overlap state is less than a second time-to-overlap with respect to a third observed spatial overlap state corresponding to the third object; and
to initiate the one or more operations to adjust the planned trajectory of the autonomous vehicle, the processing device is further to, upon determining that the third predicted spatial overlap state does not match the third observed spatial overlap state and the first time-to-overlap is less than the second time-to-overlap, associate the third erroneous prediction with a third error relevancy indicator of a third relevancy, wherein the third relevancy is greater than the second relevancy and lower than the first relevancy, and wherein the third error relevancy indicator represents a higher priority of the third erroneous prediction with respect to the second erroneous prediction and a lower priority of the third erroneous prediction with respect to the first erroneous prediction within the one or more erroneous predictions.
7 . The system of claim 1 , wherein:
the autonomous vehicle comprises a behavior prediction system that is programmed to predict object behaviors using a set of rules;
the processing device is further to use the one or more erroneous predictions to modify the set of rules to generate a modified set of rules for reprogramming the behavior prediction system; and
the one or more erroneous predictions are sorted in accordance with respective error relevancy indicators to ensure that the modified set of rules begins with rules that correspond to higher priority erroneous predictions of behavior of the one or more objects.
8 . The system of claim 1 , wherein:
the autonomous vehicle comprises a behavior prediction system that uses a trained machine learning model to predict object behaviors; and
to initiate the one or more operations to adjust the planned trajectory of the autonomous vehicle, the processing device is further to retrain the trained machine learning model using additional training data comprising the one or more erroneous predictions.
9 . The system of claim 8 , wherein the additional training data further comprises, for each erroneous prediction of the one or more erroneous predictions, a respective error relevancy indicator of the erroneous prediction.
10 . A method comprising:
identifying, by a processing device, one or more erroneous predictions of behavior for one or more objects in an environment of an autonomous vehicle traveling along a planned trajectory, wherein identifying an erroneous prediction of the one or more erroneous predictions comprises:
identifying a corresponding observed spatial overlap between the autonomous vehicle and a corresponding object;
identifying a corresponding predicted spatial overlap between the autonomous vehicle and the corresponding object;
determining a mismatch between the corresponding observed spatial overlap and the corresponding predicted spatial overlap; and
initiating, by the processing device based on the one or more erroneous predictions, one or more operations to adjust the planned trajectory of the autonomous vehicle.
11 . The method of claim 10 , wherein identifying the one or more erroneous predictions further comprises:
obtaining a set of predicted behavior data indicative of one or more predicted object behaviors for the one or more objects, and a set of observed behavior data indicative of one or more observed object behaviors of the one or more objects;
determining, based on a comparison of the set of predicted behavior data and the set of observed behavior data, whether a set of erroneous predictions exists within the set of predicted behavior data;
obtaining the set of erroneous predictions in response to determining that the set of erroneous predictions exists within the set of predicted behavior data; and
generating the one or more erroneous predictions as a subset of the set of erroneous predictions.
12 . The method of claim 11 , wherein the one or more predicted object behaviors indicate one of: that no spatial overlap is predicted between the autonomous vehicle and the corresponding object, that the autonomous vehicle arrives at a predicted spatial overlap region before the corresponding object, that the autonomous vehicle arrives at the predicted spatial overlap region after the corresponding object, or that the autonomous vehicle and the corresponding object arrive at the predicted spatial overlap region at a substantially same time.
13 . The method of claim 11 , wherein the one or more observed object behaviors indicate one of: that no spatial overlap is observed between the autonomous vehicle and the corresponding object, that the autonomous vehicle arrives at an observed spatial overlap region before the corresponding object, that the autonomous vehicle arrives at the observed spatial overlap region after the corresponding object, or that the autonomous vehicle and the corresponding object arrive at the observed spatial overlap region at a substantially same time.
14 . The method of claim 10 , further comprising:
determining, by the processing device, whether a first predicted spatial overlap state associated with a first erroneous prediction of behavior of a first object does not match a first observed spatial overlap state associated with the first erroneous prediction, wherein the first erroneous prediction is associated with a first error relevancy indicator of a first relevancy; and
upon determining that the first predicted spatial overlap state matches the first observed spatial overlap state, associating, by the processing device, a second erroneous prediction of behavior of a second object with a second error relevancy indicator of a second relevancy, the first relevancy being greater than the second relevancy, and the first error relevancy indicator representing a higher priority of the first erroneous prediction with respect to the second erroneous prediction within the one or more erroneous predictions.
15 . The method of claim 14 , further comprising:
determining, by the processing device, whether a third predicted spatial overlap state associated with a third erroneous prediction of behavior of a third object does not match a third observed spatial overlap state, and a first time-to-overlap with respect to the first observed spatial overlap state is less than a second time-to-overlap with respect to a third observed spatial overlap state corresponding to the third object; and
upon determining that the third predicted spatial overlap state does not match the third observed spatial overlap state and the first time-to-overlap is less than the second time-to-overlap, associating, by the processing device, the third erroneous prediction with a third error relevancy indicator of a third relevancy, wherein the third relevancy is greater than the second relevancy and lower than the first relevancy, and wherein the third error relevancy indicator represents a higher priority of the third erroneous prediction with respect to the second erroneous prediction and a lower priority of the third erroneous prediction with respect to the first erroneous prediction within the one or more erroneous predictions.
16 . The method of claim 10 , wherein:
the autonomous vehicle comprises a behavior prediction system that is programmed to predict object behaviors using a set of rules;
the method further comprises using, by the processing device, the one or more erroneous predictions to modify the set of rules to generate a modified set of rules for reprogramming the behavior prediction system; and
the one or more erroneous predictions are sorted in accordance with respective error relevancy indicators to ensure that the modified set of rules begins with rules that correspond to higher priority erroneous predictions of behavior of the one or more objects.
17 . The method of claim 10 , wherein:
the autonomous vehicle comprises a behavior prediction system that uses a trained machine learning model to predict object behaviors; and
initiating the one or more operations to adjust the planned trajectory of the autonomous vehicle comprises retraining the trained machine learning model using additional training data comprising the one or more erroneous predictions.
18 . The method of claim 17 , wherein the additional training data further comprises, for each erroneous prediction of the one or more erroneous predictions, a respective error relevancy indicator of the erroneous prediction.
19 . A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by a processing device, cause the processing device to:
identify one or more erroneous predictions of behavior for one or more objects in an environment of an autonomous vehicle traveling along a planned trajectory, wherein to identify an erroneous prediction of the one or more erroneous predictions, the processing device is further to:
identify a corresponding observed spatial overlap between the autonomous vehicle and a corresponding object;
identify a corresponding predicted spatial overlap between the autonomous vehicle and the corresponding object; and
determine a mismatch between the corresponding observed spatial overlap and the corresponding predicted spatial overlap; and
initiate, based on the one or more erroneous predictions, one or more operations to adjust the planned trajectory of the autonomous vehicle.
20 . The non-transitory computer-readable storage medium of claim 19 , wherein, to identify the one or more erroneous predictions, the processing device is further to:
obtain a set of predicted behavior data indicative of one or more predicted object behaviors for the one or more objects, and a set of observed behavior data indicative of one or more observed object behaviors of the one or more objects;
determine, based on a comparison of the set of predicted behavior data and the set of observed behavior data, whether a set of erroneous predictions exists within the set of predicted behavior data;
obtain the set of erroneous predictions in response to determining that the set of erroneous predictions exists within the set of predicted behavior data; and
generate the one or more erroneous predictions as a subset of the set of erroneous predictions.