Autonomous vehicle performance modeling based on controller error
Data-driven techniques are described herein for determining collision probabilities and/or predicted collision rates associated with vehicle control components used in autonomous vehicles. A performance modeling system may determine a first vehicle-distance-to-object distribution, based on log data captured in real-world driving environments. A second tracking error distribution may be received and/or determined associated with one or more vehicle control components, and the performance modeling system may determine collision probabilities and/or rates associated with the vehicle control components based on the distributions. In various examples, these techniques can be used to determine target error distributions associated with particular vehicle control components and/or particular driving maneuvers, based on overall safety and performance target metrics for an autonomous vehicle. Additionally, these techniques also can be used to predict the overall collision probability metrics for an autonomous vehicle based on tracking error distributions associated with individual vehicle control components.
1 . A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations comprising:
receiving log data associated with a vehicle operating in an environment, the log data comprising a representation of an object in the environment proximate to the vehicle;
determining, based at least in part on the log data, a first distribution representing a distribution of a distance from the object to a planned trajectory of the vehicle, wherein the first distribution is associated with a driving maneuver performed by the vehicle;
determining a second distribution representing an error distribution of a vehicle component associated with using the vehicle component to perform the driving maneuver associated with the first distribution;
determining, based at least in part on the first distribution and the second distribution, a predicted collision rate associated with performing the driving maneuver using the vehicle component; and
transmitting, based at least in part on the predicted collision rate, the vehicle component to an autonomous vehicle to cause the vehicle component to perform a vehicle control operation, wherein the vehicle control operation comprises one or more of: an acceleration control operation, a braking control operation, or a steering maneuver control operation.
2 . The system of claim 1 , wherein the driving maneuver comprises at least one of:
a stopping driving maneuver;
a turning driving maneuver;
a single lane change driving maneuver; or
a double lane change driving maneuver.
3 . The system of claim 2 , wherein the driving maneuver further comprises at least one of:
a speed associated with the driving maneuver; or
a driving condition associated with the driving maneuver.
4 . The system of claim 1 , wherein determining the second distribution comprises:
determining a vehicle position error associated with the driving maneuver performed by at least one of a test vehicle or a simulated vehicle operating the vehicle component.
5 . The system of claim 1 , wherein the second distribution is determined based at least in part on a target collision rate associated with the autonomous vehicle.
6 . A method comprising:
receiving log data associated with a vehicle operating in an environment, the log data comprising a representation of an object in the environment proximate to the vehicle;
determining, based at least in part on the log data, a first distribution representing a distribution of a distance from the object to a planned trajectory of the vehicle, wherein the first distribution is associated with a driving maneuver performed by the vehicle;
determining a second distribution representing an error distribution of a vehicle component associated with using the vehicle component to perform the driving maneuver associated with the first distribution;
determining, based at least in part on the first distribution and the second distribution, a predicted event rate associated with the vehicle component; and
transmitting, based at least in part on the predicted event rate, the vehicle component to an autonomous vehicle to cause the vehicle component to perform a vehicle control operation, wherein the vehicle control operation comprises one or more of: an acceleration control operation, a braking control operation, or a steering maneuver control operation.
7 . The method of claim 6 , wherein the second distribution comprises at least one of:
a lateral position error distribution; or
a heading error distribution, and
wherein determining the first distribution comprises determining, based on the log data, a distribution of lateral distances between the planned trajectory of the vehicle and a plurality of objects represented in the log data.
8 . The method of claim 6 , wherein the second distribution comprises at least one of:
a longitudinal position error distribution;
a longitudinal stopping distance error distribution; or
a longitudinal velocity error distribution, and
wherein determining the first distribution comprises determining, based on the log data, a distribution of longitudinal distances between the planned trajectory of the vehicle and a plurality of objects represented in the log data.
9 . The method of claim 6 , wherein determining the first distribution comprises:
determining a predicted perception error rate associated with a perception component of the vehicle; and
modifying the first distribution based at least in part on the predicted perception error rate.
10 . The method of claim 6 , wherein determining the second distribution comprises:
determining a vehicle position error associated with the driving maneuver performed by at least one of a test vehicle or a simulated vehicle operating the vehicle component.
11 . The method of claim 6 , wherein the second distribution is determined based at least in part on a target event rate associated with the autonomous vehicle.
12 . The method of claim 6 , wherein determining the first distribution comprises determining, based on the log data, a distribution of distances between the planned trajectory of the vehicle and a plurality of objects within environments that include a first driving condition.
13 . The method of claim 6 , wherein the second distribution comprises determining a joint distribution based at least in part on:
a first lateral position error distribution; and
a second longitudinal position error distribution,
and wherein determining the first distribution comprises determining, based on the log data, a distribution of total distances between the planned trajectory of the vehicle and a plurality of objects represented in the log data.
14 . The method of claim 6 , further comprising:
determining a performance degradation associated with operation of the autonomous vehicle; and
determining vehicle component performance data, based at least in part on the performance degradation and the second distribution,
wherein a trajectory planner component is configured to determine a trajectory for controlling the autonomous vehicle, based at least in part on the vehicle component performance data.
15 . One or more non transitory computer readable media storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:
receiving log data associated with a vehicle operating in an environment, the log data comprising a representation of an object in the environment proximate to the vehicle;
determining, based at least in part on the log data, a first distribution representing a distribution of a distance from the object to a planned trajectory of the vehicle, wherein the first distribution is associated with a driving maneuver performed by the vehicle;
determining a second distribution representing an error distribution of a vehicle component associated with using the vehicle component to perform the driving maneuver associated with the first distribution;
determining, based at least in part on the first distribution and the second distribution, a predicted event rate associated with the vehicle component; and
transmitting, based at least in part on the predicted event rate, the vehicle component to an autonomous vehicle to cause the vehicle component to perform a vehicle control operation, wherein the vehicle control operation comprises one or more of: an acceleration control operation, a braking control operation, or a steering maneuver control operation.
16 . The one or more non transitory computer readable media of claim 15 , wherein the second distribution comprises at least one of:
a lateral position error distribution; or
a heading error distribution, and
wherein determining the first distribution comprises determining, based on the log data, a distribution of lateral distances between the planned trajectory of the vehicle and a plurality of objects represented in the log data.
17 . The one or more non transitory computer readable media of claim 15 , wherein the second distribution comprises at least one of:
a longitudinal position error distribution;
a longitudinal stopping distance error distribution; or
a longitudinal velocity error distribution, and
wherein determining the first distribution comprises determining, based on the log data, a distribution of longitudinal distances between the planned trajectory of the vehicle and a plurality of objects represented in the log data.
18 . The one or more non transitory computer readable media of claim 15 , wherein determining the first distribution comprises:
determining a predicted perception error rate associated with a perception component of the vehicle; and
modifying the first distribution based at least in part on the predicted perception error rate.
19 . The one or more non transitory computer readable media of claim 15 , wherein determining the second distribution comprises:
determining a vehicle position error associated with the driving maneuver performed by at least one of a test vehicle or a simulated vehicle operating the vehicle component.
20 . The one or more non transitory computer readable media of claim 15 , wherein the second distribution is determined based at least in part on a target event rate associated with the autonomous vehicle.