Metric learning prediction of simulation parameters
Predicting simulation parameters is performed by obtaining a plurality of datasets, each dataset including simulation parameters, time series data and a label. Wherein the time series data represents a simulation status for each time and the label represents a simulation result. Learning a metric of the simulation parameters including two datasets of the plurality of datasets. Wherein the metric imitates the similarity of time series data of the two datasets, and training a model for predicting the label for simulation parameters by using the metric.
1 . A computer-implemented method comprising:
increasing prediction accuracy while reducing computational resource consumption of a system under test (SUT) for automated driving vehicles by iteratively training the SUT to adapt to real-world traffic environments and obtain a trained SUT by:
obtaining a plurality of datasets, each dataset including previously predicted simulation parameters obtained by prediction without simulation, time series data and a label relating to autonomous vehicle operation, wherein the time series data represents a simulation status for each time step and the label represents a simulation result;
learning a metric of the previously predicted simulation parameters of at least two datasets of the plurality of datasets, by determining, with a neural network model, a weight matrix that can minimize a total absolute error between distances between sets of simulation parameters for different datasets and a similarity of the time series data of the different datasets through dynamic time warping by utilizing one or more regularization terms to regularize matrix elements of the weight matrix, wherein the metric imitates the similarity of the time series data;
training the neural network model of the SUT to predict a label for car driving simulation parameters without performing a current simulation by:
mapping previously predicted simulation parameters of the at least two datasets to labels using a radial basis function kernel based on the metric and a Gaussian process as a response-surface model;
searching for a preferable value of a scaling factor among multiple settings based on a prediction accuracy of the neural network model;
determining, with the neural network model, new car driving simulation parameters that causes the SUT of the automated driving vehicles to output one or more labels for the car driving simulation parameters that represent degrees of risk of an undesirable event for the automated driving vehicles involving movement of other vehicles in a traffic scene based on the label predicted for the car driving simulation parameters; and
controlling, with the trained SUT, a vehicle speed system and a vehicle steering system to respectively control a speed and a direction of an automated driving vehicle to react to the degrees of risk and avoid the undesirable event based on the new car driving simulation parameters responsive to one or more labels output from the trained neural network model that correspond to behaviors in real-world traffic environments.
2 . The computer-implemented method of claim 1 , wherein the metric is calculated based on a difference of simulation parameters of the at least two datasets and a set of weights, and
the learning updates the set of weights to imitate the similarity of time series data of the at least two datasets.
3 . The computer-implemented method of claim 1 , further comprising determining new simulation parameters that maximize the label predicted by the neural network model.
4 . The computer-implemented method of claim 1 , further comprising determining new simulation parameters that minimize the label predicted by the neural network model.
5 . The computer-implemented method of claim 1 , further comprising determining new simulation parameters by selecting the new simulation parameters from at least two sets of candidate simulation parameters.
6 . The computer-implemented method of claim 1 , further comprising:
determining new simulation parameters by using the model;
performing a simulation for the new simulation parameters to obtain new time series data and a new label for the new simulation parameters; and
adding a new dataset including the new simulation parameters, the new time series data and the new label to the plurality of datasets.
7 . The computer-implemented method of claim 6 , wherein the previously predicted simulation parameters represent initial parameters of at least two vehicles, and the time series data represents reacting results of the at least two vehicles during movements of the at least two vehicles including speeds and directions of the at least two vehicles.
8 . The computer-implemented method of claim 7 , wherein the determining includes determining the new simulation parameters that cause the model to predict a label representing a multi-reacting collision.
9 . The computer-implemented method of claim 7 , wherein the at least two vehicles include an automated driving vehicle, and
the performing of the simulation includes simulating behaviors of the automated driving vehicle.
10 . A computer program product including one or more non-transitory computer readable storage mediums collectively storing program instructions that are executable by a processor or programmable circuitry to cause the processor or programmable circuitry to perform operations comprising:
increasing prediction accuracy while reducing computational resource consumption of a system under test (SUT) for automated driving vehicles by iteratively training the SUT to adapt to real-world traffic environments and obtain a trained SUT by:
obtaining a plurality of datasets, each dataset including previously predicted simulation parameters, time series data and a label relating to autonomous vehicle operation, wherein the time series data represents a simulation status for each time step and the label represents a simulation result;
learning a metric of the previously predicted simulation parameters of at least two datasets of the plurality of datasets, by determining, with a neural network model, a weight matrix that can minimize a total absolute error between distances between sets of simulation parameters for different datasets and a similarity of the time series data of the different datasets through dynamic time warping by utilizing one or more regularization terms to regularize matrix elements of the weight matrix, wherein the metric imitates the similarity of the time series data;
training the neural network model of the SUT to predict a label for car driving simulation parameters without performing a current simulation by:
mapping previously predicted simulation parameters of the at least two datasets to labels using a radial basis function kernel based on the metric and a Gaussian process as a response-surface model;
searching for a preferable value of a scaling factor among multiple settings based on a prediction accuracy of the neural network model;
determining, with the neural network model, new car driving simulation parameters that causes the SUT of the automated driving vehicles to output one or more labels for the car driving simulation parameters that represent degrees of risk of an undesirable event for the automated driving vehicles involving movement of other vehicles in a traffic scene based on the label predicted for the car driving simulation parameters; and
controlling, with the trained SUT, a vehicle speed system and a vehicle steering system to respectively control a speed and a direction of an automated driving vehicle to react to the degrees of risk and avoid the undesirable event based on the new car driving simulation parameters to one or more labels output from the trained neural network model that correspond to behaviors in real-world traffic environments.
11 . The computer program product of claim 10 , wherein the metric is calculated based on a difference of simulation parameters of the at least two datasets and a set of weights, and
the learning updates the set of weights to imitate the similarity of time series data of the at least two datasets.
12 . The computer program product of claim 10 , wherein the operations further comprising determining new simulation parameters that maximize the label predicted by the trained neural network model.
13 . The computer program product of claim 10 , wherein the operations further comprising:
determining new simulation parameters by using the trained neural network model;
performing a simulation for the new simulation parameters to obtain new time series data and a new label for the new simulation parameters; and
adding a new dataset including the new simulation parameters, the new time series data and the new label to the plurality of datasets.
14 . An apparatus comprising:
a processor or a programmable circuitry; and
one or more computer readable mediums collectively including instructions that, when executed by the processor or the programmable circuitry, cause the processor or the programmable circuitry to:
increase prediction accuracy while reducing computational resource consumption of a system under test (SUT) for automated driving vehicles by iteratively training the SUT to adapt to real-world traffic environments and obtain a trained SUT by:
obtain a plurality of datasets, each dataset including previously predicted simulation parameters, time series data and a label relating to autonomous vehicle operation, wherein the time series data represents a simulation status for each time step and the label represents a simulation result;
learn a metric of the previously predicted simulation parameters of at least two datasets of the plurality of datasets, by determining, with a neural network model, a weight matrix that can minimize a total absolute error between distances between sets of simulation parameters for different datasets and a similarity of the time series data of the different datasets through dynamic time warping by utilizing one or more regularization terms to regularize matrix elements of the weight matrix, wherein the metric imitates the similarity of the time series data;
train the neural network model of the SUT to predict a label for car driving simulation parameters without performing a current simulation by:
mapping previously predicted simulation parameters of the at least two datasets to labels using a radial basis function kernel based on the metric and a Gaussian process as a response-surface model;
searching for a preferable value of a scaling factor among multiple settings based on a prediction accuracy of the neural network model;
determine, with the neural network model, new car driving simulation parameters that causes the SUT of the automated driving vehicles to output one or more labels for the car driving simulation parameters that represent degrees of risk of an undesirable event for the automated driving vehicles involving movement of other vehicles in a traffic scene based on the label predicted for the car driving simulation parameters; and
control, with the trained SUT, a vehicle speed system and a vehicle steering system to respectively control a speed and a direction of an automated driving vehicle to react to the degrees of risk and avoid the undesirable event based on the new car driving simulation parameters responsive to one or more labels output from the trained neural network model that correspond to behaviors in real-world traffic environments.
15 . The apparatus of claim 14 , wherein the metric is calculated based on a difference of simulation parameters of the at least two datasets and a set of weights, and
the learning updates the set of weights to imitate the similarity of time series data of the at least two datasets.
16 . The apparatus of claim 14 , wherein the instructions further cause the processor or the programmable circuitry to determine new simulation parameters that maximize the label predicted by the trained neural network model.
17 . The apparatus of claim 14 , wherein the instructions further cause the processor or the programmable circuitry to:
determine new simulation parameters by using the trained neural network model;
perform a simulation for the new simulation parameters to obtain new time series data and a new label for the new simulation parameters; and
add a new dataset including the new simulation parameters, the new time series data and the new label to the plurality of datasets.