Vehicle control combining neural network and physics prediction
In exemplary embodiments, methods and systems are provided that include one or more sensors of a vehicle and a processor of the vehicle. The one or more sensors are configured to obtain sensor data as to operation of the vehicle. The processor is coupled to the one or more sensors, and is configured to at least facilitate inputting the sensor data into a grey-box model including a neural network model for the forces acting on the vehicle and a physics-based model for body motion of the vehicle; determining a trajectory of the vehicle, using the grey-box model; determining a control action for the vehicle, based on the grey-box model using the sensor data; and executing the control action for the vehicle, in accordance with instructions provided by the processor.
1 . A method comprising:
obtaining, via one or more sensors of a vehicle, sensor data as to operation of the vehicle;
inputting the sensor data into a grey-box model comprising a neural network model for the forces acting on the vehicle and a physics-based model for body motion of the vehicle, via a processor of the vehicle;
determining, via the processor, a trajectory of the vehicle, using the grey-box model;
determining, via the processor, a control action for the vehicle, based on the grey-box model using the sensor data; and
executing the control action for the vehicle, in accordance with instructions provided by the processor;
wherein the neural network model comprises:
an input layer including input features from the sensor data that include each of the following: driving and braking torques, normal loads, slip angles, chassis velocities, and yaw rate as determined by an inertial measurement unit (IMU) of the one or more sensors;
a plurality of processing layers; and
an output layer; and
wherein the neural network model and the physics-based model are utilized together as a hybrid model by the processor in generating outputted results of an initial state of the vehicle using both the physics-based model along with ground forces between the vehicle and a ground of a roadway in which the vehicle is travelling to generate the outputted results, and the outputted results are utilized by the processor in executing a model predictive control (MPC) for optimizing performance criteria for the neural network model for a predetermined number of steps in a cyclic manner and for executing the control action for movement of the vehicle corresponding to a receding horizon control for the vehicle.
2 . The method of claim 1 , wherein the control action includes one or more of the following: application of a steering angle for the vehicle, application of a braking torque for the vehicle, or application of a propulsion motor torque for the vehicle.
3 . The method of claim 2 , wherein the control action includes each of the following: application of the steering angle for the vehicle, application of the braking torque for the vehicle, and application of the propulsion motor torque for the vehicle.
4 . The method of claim 1 , wherein:
the neural network model pertains to vehicle to ground interaction between a plurality of wheels of the vehicle and a ground of a roadway on which the vehicle is travelling; and
the neural network model is updated based on application of the physics-based model.
5 . The method of claim 4 , wherein the neural network model includes a Jacobian matrix that comprises a gradient matrix that is configured to improve efficiency for the neural network model and therefore to improve control of the vehicle.
6 . The method of claim 4 , wherein the neural network model is trained used the grey-box model along with vehicle simulation data and vehicle test data based on a loss function including separate equations for lateral acceleration and longitudinal acceleration.
7 . The method of claim 6 , further comprising developing model based control for the vehicle, based on a linear time varying (LTV) approximation of the grey-box model.
8 . The method of claim 4 , wherein the physics-based model is implemented by applying torque to each of four corners of the vehicle, via application of the torque to respective wheels at each of the respective corners of the vehicle, and taking outputs of the neural network model and estimates values of a longitudinal acceleration, a lateral acceleration, and a yaw acceleration of the vehicle using one or more model-based control methods comprising one or more model predictive control (MPC) methods, linear quadratic regulator (LQR) methods, or iterative linear quadratic regulator (iLQR) methods.
9 . The method of claim 4 , wherein the physics-based model is implemented by applying torque to each of four corners of the vehicle, via application of the torque to respective wheels at each of the respective corners of the vehicle, and taking outputs of the neural network model and estimates values of a longitudinal acceleration, a lateral acceleration, and a yaw acceleration of the vehicle using one or more model-based control methods comprising each of the following: one or more model predictive control (MPC) methods, one or more linear quadratic regulator (LQR) methods, and one or more iterative linear quadratic regulator (iLQR) methods.
10 . A system comprising:
one or more sensors of a vehicle that are configured to obtain sensor data as to operation of the vehicle, the one or more sensors including an inertial measurement unit (IMU); and
a processor of the vehicle, the processor coupled to the one or more sensors and configured to at least facilitate:
inputting the sensor data into a grey-box model comprising a neural network model for the forces acting on the vehicle and a physics-based model for body motion of the vehicle;
determining a trajectory of the vehicle, using the grey-box model;
determining a control action for the vehicle, based on the grey-box model using the sensor data; and
executing the control action for the vehicle, in accordance with instructions provided by the processor;
wherein the neural network model comprises:
an input layer including input features from the sensor data that include each of the following: driving and braking torques, normal loads, slip angles, chassis velocities, and yaw rate as determined by the IMU;
a plurality of processing layers; and
an output layer; and
wherein the processor is further configured to utilize the neural network model and the physics-based model together as a hybrid model by the processor in generating outputted results of an initial state of the vehicle using both the physics-based model along with ground forces between the vehicle and a ground of a roadway in which the vehicle is travelling to generate the outputted results, and to utilize the outputted results in executing a model predictive control (MPC) for optimizing performance criteria for the neural network model for a predetermined number of steps in a cyclic manner and for executing the control action for movement of the vehicle corresponding to a receding horizon control for the vehicle.
11 . The system of claim 10 , wherein the control action includes one or more of the following: application of a steering angle for the vehicle, application of a braking torque for the vehicle, or application of a propulsion motor torque for the vehicle.
12 . The system of claim 11 , wherein the control action includes each of the following: application of the steering angle for the vehicle, application of the braking torque for the vehicle, and application of the propulsion motor torque for the vehicle.
13 . The system of claim 10 , wherein: the neural network model pertains to vehicle to ground interaction between a plurality of wheels of the vehicle and a ground of a roadway on which the vehicle is travelling; and the neural network model is updated based on application of the physics-based model.
14 . The system of claim 13 , wherein the neural network model includes a Jacobian matrix that comprises a gradient matrix that is configured to improve efficiency for the neural network model and therefore to improve control of the vehicle.
15 . The system of claim 13 , wherein the neural network model is trained used the grey-box model along with vehicle simulation data and vehicle test data based on a loss function including separate equations for lateral acceleration and longitudinal acceleration.
16 . The system of claim 15 , wherein the processor is further configured to at least facilitate developing model based control for the vehicle, based a linear time varying (LTV) approximation of the grey-box model.
17 . The system of claim 13 , wherein the processor is further configured to at least facilitate implementing the physics-based model by applying torque to each of four corners of the vehicle, via application of the torque to respective wheels at each of the respective corners of the vehicle, and taking outputs of the neural network model and estimates values of a longitudinal acceleration, a lateral acceleration, and a yaw acceleration of the vehicle using one or more model-based control methods comprising one or more model predictive control (MPC) methods, linear quadratic regulator (LQR) methods, or iterative linear quadratic regulator (iLQR) methods.
18 . The system of claim 13 , wherein the processor is further configured to at least facilitate implementing the physics-based model by applying torque to each of four corners of the vehicle, via application of the torque to respective wheels at each of the respective corners of the vehicle, and taking outputs of the neural network model and estimates values of a longitudinal acceleration, a lateral acceleration, and a yaw acceleration of the vehicle using one or more model-based control methods comprising each of the following: one or more model predictive control (MPC) methods, one or more linear quadratic regulator (LQR) methods, and one or more iterative linear quadratic regulator (iLQR) methods.
19 . A vehicle comprising:
a body;
a drive system, steering system, and braking system configured to control movement of the body;
one or more sensors that are configured to obtain sensor data as to operation of the vehicle, the one or more sensors including an inertial measurement unit (IMU); and
a processor that is coupled to the one or more sensors and configured to at least facilitate:
inputting the sensor data into a grey-box model comprising a neural network model for the forces acting on the vehicle and a physics-based model for body motion of the vehicle;
determining a trajectory of the vehicle, using the grey-box model;
determining a control action for movement of the vehicle via one or more of the drive system, steering system, and braking system, based on the grey-box model using the sensor data; and
executing the control action for the vehicle, in accordance with instructions provided by the processor;
wherein the neural network model comprises:
an input layer including input features from the sensor data that include each of the following: driving and braking torques, normal loads, slip angles, chassis velocities, and yaw rate as determined by the IMU;
a plurality of processing layers; and
an output layer; and
wherein the processor is further configured to utilize the neural network model and the physics-based model together as a hybrid model by the processor in generating outputted results of an initial state of the vehicle using both the physics-based model along with ground forces between the vehicle and a ground of a roadway in which the vehicle is travelling to generate the outputted results, and to utilize the outputted results in executing a model predictive control (MPC) for optimizing performance criteria for the neural network model for a predetermined number of steps in a cyclic manner and for executing the control action for movement of the vehicle corresponding to a receding horizon control for the vehicle.
20 . The vehicle of claim 19 , wherein:
the neural network model pertains to vehicle to ground interaction between a plurality of wheels of the vehicle and a ground of a roadway on which the vehicle is travelling;
the neural network model updated based on application of the physics-based model;
the neural network model includes a Jacobian matrix that comprises a gradient matrix that is configured to improve efficiency for the neural network model and therefore to improve control of the vehicle;
the neural network model is trained used the grey-box model along with vehicle simulation data and vehicle test data based on a loss function including separate equations for lateral acceleration and longitudinal acceleration;
the processor is further configured to at least facilitate model based control for the vehicle, based a linear time varying (LTV) approximation of the grey-box model; and
the processor is further configured to at least facilitate implementing the physics-based model by applying torque to each of four corners of the vehicle, via application of the torque to respective wheels at each of the respective corners of the vehicle, and taking outputs of the neural network model and estimates values of a longitudinal acceleration, a lateral acceleration, and a yaw acceleration of the vehicle using one or more model-based control methods comprising one or more model predictive control (MPC) methods, linear quadratic regulator (LQR) methods, or iterative linear quadratic regulator (iLQR) methods.