SYSTEM AND METHOD FOR CONTROLLING MACHINE LEARNING-BASED VEHICLES
A control device is used in a vehicle including a perception system which uses sensors. The perception system includes a device for estimating a variable including a characteristic relating to objects detected in the surrounding area of the vehicle, the estimation device including an online learning module which uses a neural network to estimate the variable. The learning module includes: a forward-propagation module to propagate data from sensors, which data are applied as the input to the neural network, so as to provide a predicted output including an estimate of the variable; a fusion system to determine a fusion output by implementing a sensor fusion algorithm using the predicted values; a back-propagation module to update weights associated with the online neural network by determining a loss function representing the error between an improved predicted value of the fusion output and the predicted output by performing gradient descent back propagation.
1 - 11 . (canceled)
12 . A control device implemented in a vehicle, the vehicle comprising a perception system using a set of sensors, each sensor providing data, the perception system comprising an estimation device configured to estimate a variable comprising at least one feature in relation to one or more objects detected in an environment of the vehicle, the estimation device comprising an online learning module using a neural network to estimate said variable, the neural network being associated with a set of weights, the learning module comprising:
a forward propagation module configured to propagate data from one or more sensors applied at an input of the neural network, so as to provide a predicted output comprising an estimation of said variable;
a fusion system configured to determine a fusion output by implementing at least one sensor fusion algorithm based on at least some of said predicted values; and
a backpropagation module configured to update the weights associated with the neural network online by determining a loss function representing an error between an improved predicted value of said fusion output and said predicted output by performing a gradient descent backpropagation.
13 . The device as claimed in claim 12 , wherein said variable is a state vector comprising information in relation to the position and/or the movement of an object detected by the perception system.
14 . The device as claimed in claim 13 , wherein said state vector further comprises information in relation to one or more detected objects.
15 . The device as claimed in claim 14 , wherein said state vector further comprises trajectory parameters of a target object.
16 . The device as claimed in claim 12 , wherein said improved predicted value is determined by applying a Kalman filter.
17 . The device as claimed in claim 12 , further comprising a replay buffer configured to store the outputs predicted by the estimation device and/or the fusion outputs delivered by the fusion system.
18 . The device as claimed in claim 17 , further comprising a recurrent neural network encoder configured to encode and compress the data prior to storage in the replay buffer, and a decoder configured to decode and decompress the data extracted from the replay buffer.
19 . The device as claimed in claim 18 , wherein the encoder is a recurrent neural network encoder and the decoder is a recurrent neural network decoder.
20 . The device as claimed in claim 17 , wherein the replay buffer is prioritized.
21 . The device as claimed in claim 17 , wherein the device is configured to implement a condition for testing input data applied at input of a neural network, input data being deleted from the replay buffer when the loss function between the value predicted for this input sample and the fusion output is lower than a predefined threshold.
22 . A control method implemented in a vehicle, the vehicle comprising a perception system using a set of sensors, each sensor providing data, the control method comprising:
estimating a variable comprising at least one feature in relation to one or more objects detected in an environment of the vehicle, wherein the estimating implements online learning step a neural network to estimate said variable, the neural network being associated with a set of weights,
wherein the online learning comprises:
propagating data from one or more sensors, applied at an input of the neural network, so as to provide a predicted output comprising an estimation of said variable;
determining a fusion output by implementing at least one sensor fusion algorithm based on at least some of said predicted values; and
updating the weights associated with the neural network online by determining a loss function representing an error between an improved predicted value of said fusion output and said predicted output by performing a gradient descent backpropagation.