Safety analysis framework
Techniques for determining a safety metric associated with a vehicle controller are discussed herein. To determine whether a complex system (which may be uninspectable) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data and associated with a scenario parameter to be adjusted. To validate safe operation of such a system, a scenario may be identified for inspection. Error metrics of a subsystem of the system can be quantified. The error metrics, in addition to stochastic errors of other systems/subsystems can be introduced to the scenario. The scenario parameter may also be perturbed. Any multitude of such perturbations can be instantiated in a simulation to test, for example, a vehicle controller. A safety metric associated with the vehicle controller can be determined based on the simulation, as well as causes for any failures.
1. A system comprising:
one or more processors; and
one or more computer-readable media storing computer-executable instructions that, when executed, cause the system to perform operations comprising:
receiving log data associated with operating an autonomous vehicle in an environment;
determining, based at least in part on the log data, a set of scenarios, a scenario of the set of scenarios comprising a scenario parameter associated with an aspect of the environment;
determining a plurality of error models associated with a subsystem of the autonomous vehicle, an individual error model of the plurality of error models indicating an error and an error distribution associated with the subsystem of the autonomous vehicle;
identifying, based at least in part on the scenario parameter, an error model of the plurality of error models;
determining, based at least in part on the scenario parameter and the error model, a parameterized scenario, wherein the parameterized scenario represents a possible variation of the scenario;
perturbing the parameterized scenario by adding an error indicated by the error model to at least one of a component of a simulated vehicle to be instantiated in a perturbed parameterized scenario or the scenario parameter, the simulated vehicle being controlled by a vehicle controller;
instantiating the simulated vehicle in the perturbed parameterized scenario;
receiving simulation data indicating how the simulated vehicle responds to the perturbed parameterized scenario; and
determining, based at least in part on the simulation data, a safety metric associated with the vehicle controller, wherein the safety metric represents an outcome associated with the parameterized scenario.
2. The system of claim 1 , wherein determining the set of scenarios comprises:
clustering the log data to determine a first set of clusters, wherein an individual cluster of the first set of clusters is associated with an individual scenario;
determining, based at least in part on the first set of clusters, a probability associated with the individual cluster; and
determining, based at least in part on a probability threshold and the first set of clusters, a second set of clusters.
3. The system of claim 1 , wherein determining the plurality of error models comprises:
receiving ground truth data associated with the environment;
determining, based at least in part on comparing the ground truth data to the log data, an error; and
determining, based at least in part on the error, an error distribution.
4. The system of claim 1 , wherein the parameterized scenario is a first parameterized scenario, the perturbed parameterized scenario is a first perturbed parameterized scenario, and the simulation data is first simulation data, the operations further comprising:
determining, based on the first simulation data, a second parameterized scenario comprising at least one of a first subset of the scenario parameter or a second subset of the error model;
perturbing the second parameterized scenario as a second perturbed parameterized scenario;
instantiating the simulated vehicle in the second perturbed parameterized scenario;
receiving second simulation data; and
updating, based at least in part on the second simulation data, the safety metric.
5. A method comprising:
determining a scenario comprising a scenario parameter describing a portion of an environment;
receiving a plurality of error models associated with a subsystem of a vehicle;
identifying, based at least in part on the scenario parameter, an error model of the plurality of error models;
determining, based at least in part on the scenario, the scenario parameter, and the error model, a parameterized scenario, wherein the parameterized scenario represents a possible variation of the scenario;
perturbing the parameterized scenario as a perturbed parameterized scenario by adding an error indicated by the error model;
receiving simulation data indicating how the subsystem of the vehicle responds to the perturbed parameterized scenario; and
determining, based at least in part on the simulation data, a safety metric associated with the subsystem of the vehicle, wherein the safety metric represents an outcome associated with the parameterized scenario.
6. The method of claim 5 , wherein the scenario parameter is associated with at least one of an object size, an object velocity, an object pose, an object density, a vehicle velocity, a vehicle trajectory.
7. The method of claim 5 , wherein determining the scenario comprises:
receiving log data associated with an autonomous vehicle;
clustering the log data to determine a first set of clusters, wherein an individual cluster of the first set of clusters is associated with the scenario;
determining, based at least in part on the first set of clusters, a probability associated with the individual cluster; and
determining that the probability meets or exceeds a probability threshold.
8. The method of claim 5 , wherein the error model is identified based at least in part on:
receiving ground truth data associated with the environment;
determining, based at least in part on comparing the ground truth data to log data associated with the vehicle, an error; and
determining, based at least in part on the error, an error distribution;
wherein the error model comprises the error distribution.
9. The method of claim 5 , wherein the parameterized scenario is a first parameterized scenario, the perturbed parameterized scenario is a first perturbed parameterized scenario, and the simulation data is first simulation data, the method further comprising:
determining, based on the first simulation data, a second parameterized scenario comprising at least one of a first subset of the scenario parameter or a second subset of the error model;
perturbing the second parameterized scenario;
receiving second simulation data; and
updating, based at least in part on the second simulation data, the safety metric.
10. The method of claim 9 , further comprising:
disabling at least a first portion of one of the scenario parameter or the error model; and
associating the second simulation data with at least a second portion of one of the scenario parameter or the error model that is not disabled.
11. The method of claim 5 , wherein the portion is a first portion, the method further comprising:
receiving map data, wherein a second portion of the map data is associated with the first portion of the environment; and
determining that the second portion of the map data is associated with a scenario associated with a probability that meets or exceeds a threshold probability associated with the scenario parameter.
12. A non-transitory computer-readable medium storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:
determining a scenario comprising a scenario parameter describing a portion of an environment;
one or more of receiving or determining a plurality of error models associated with a subsystem of a vehicle;
identifying, based at least in part on the scenario parameter, an error model of the plurality of error models;
determining, based at least in part on the scenario, the scenario parameter, and the error model, a parameterized scenario, wherein the parameterized scenario represents a possible variation of the scenario;
perturbing the parameterized scenario as a perturbed parameterized scenario by adding an error indicated by the error model;
receiving simulation data indicating how the subsystem of the vehicle responds to the perturbed parameterized scenario; and
determining, based at least in part on the simulation data, a safety metric associated with the subsystem of the vehicle, wherein the safety metric represents an outcome associated with the parameterized scenario.
13. The non-transitory computer-readable medium of claim 12 , wherein the scenario parameter is associated with at least one of an object size, an object velocity, an object pose, an object density, a vehicle velocity, a vehicle trajectory.
14. The non-transitory computer-readable medium of claim 12 , wherein determining the scenario comprises:
receiving log data associated with an autonomous vehicle;
clustering the log data to determine a first set of clusters, wherein an individual cluster of the first set of clusters is associated with the scenario;
determining, based at least in part on the first set of clusters, a probability associated with the individual cluster; and
determining that the probability meets or exceeds a probability threshold.
15. The non-transitory computer-readable medium of claim 12 , wherein the error model is identified based at least in part on:
receiving ground truth data associated with the environment;
determining, based at least in part on comparing the ground truth data to log data associated with the vehicle, an error; and
determining, based at least in part on the error, an error distribution;
wherein the error model comprises the error distribution.
16. The non-transitory computer-readable medium of claim 12 , wherein the parameterized scenario is a first parameterized scenario, the perturbed parameterized scenario is a first perturbed parameterized scenario, and the simulation data is first simulation data, the operations further comprising:
determining, based on the first simulation data, a second parameterized scenario comprising at least one of a first subset of the scenario parameter or a second subset of the error model;
perturbing the second parameterized scenario;
receiving second simulation data; and
updating, based at least in part on the second simulation data, the safety metric.
17. The non-transitory computer-readable medium of claim 16 , the operations further comprising:
disabling at least a first portion of one of the scenario parameter or the error model; and
associating the second simulation data with at least a second portion of one of the scenario parameter or the error model that is not disabled.
18. The non-transitory computer-readable medium of claim 12 , wherein the safety metric indicates a probability of meeting or exceeding a cost threshold.
19. The non-transitory computer-readable medium of claim 12 , wherein the error model is associated with one or more of a perception system of the vehicle, a prediction system of the vehicle, or a planner system of the vehicle.
20. The method of claim 5 , further comprising:
determining the parameterized scenario further based at least in part on a scenario probability representing a probability of encountering the scenario in the environment.