Method and system for controlling a vehicle using machine learning
A method for controlling a vehicle includes: determining an accelerator position sensor/brake pedal position sensor (APS/BPS) command value based on a state variable and a reward variable including a prediction value for a future velocity of the vehicle predicted based on a past APS/BPS command value of the vehicle; and learning for a reward value according to the reward variable to satisfy a predetermined goal based on a change that the determined APS/BPS command value causes to at least one state variable under given environment information.
1. A method for controlling a vehicle, the method comprising:
determining an accelerator position sensor/brake pedal position sensor (APS/BPS) command value based on a state variable and a reward variable; and
learning a reward value according to the reward variable to satisfy a predetermined goal based on a change that the determined APS/BPS command value causes to the state variable under given environment information, wherein the state variable includes a prediction value for a future velocity of the vehicle predicted based on a past APS/BPS command value of the vehicle.
2. The method of claim 1 , wherein the reward variable comprises an index determining a relative ratio contributing to satisfying the predetermined goal between an error of a current velocity and an error of the future velocity.
3. The method of claim 1 , further comprising:
generating the APS/BPS command value for following a predetermined velocity profile with a driver agent to which a result of learning is applied; and
controlling a power device of the vehicle based on the APS/BPS command value.
4. The method of claim 1 , wherein determining the APS/BPS command value comprises:
determining an APS/BPS variation based on the state variable and the reward variable, and
determining the APS/BPS command value based on the APS/BPS variation and the APS/BPS command value at a time point ahead of a predetermined time from a current time point.
5. The method of claim 1 , wherein the prediction value for the future velocity of the vehicle is determined based on past velocity information of the vehicle corresponding to at least two different time points.
6. The method of claim 1 , wherein the prediction value for the future velocity of the vehicle is determined based on i) a value obtained by passing past acceleration information of the vehicle through a low pass filter and ii) a current velocity of the vehicle.
7. The method of claim 1 , wherein the past APS/BPS command value is a past APS/BPS value during a predetermined period.
8. The method of claim 1 , wherein the prediction value for the future velocity comprises a prediction value for the future velocity during a predetermined period.
9. The method of claim 1 , wherein the state variable comprises at least one of a past velocity of the vehicle, an error of an actual velocity with respect to a past target velocity of the vehicle, or a target velocity of the vehicle.
10. The method of claim 9 , wherein the target velocity of the vehicle has different values depending on a driving mode of the vehicle.
11. A system for controlling a vehicle, the system comprising:
a vehicle velocity predictor configured to generate a prediction value for a future velocity predicted based on a past accelerator position sensor/brake pedal position sensor (APS/BPS) command value of the vehicle; and
an agent implementation unit configured to:
determine an APS/BPS command value based on a state variable and a reward variable, and
learn a reward value according to the reward variable to satisfy a predetermined goal based on a change that the determined APS/BPS command value gives to the state variable under given environment information, wherein the state variable includes a prediction value for a future velocity of the vehicle predicted based on a past APS/BPS command value of the vehicle.
12. The system of claim 11 , wherein the reward variable comprises an index determining a relative ratio contributing to satisfying the predetermined goal between an error of a current velocity and an error of the future velocity.
13. The system of claim 11 , further comprising a power device controlled based on an APS/BPS command value,
wherein the agent implementation unit is configured to generate the APS/BPS command value for following a predetermined velocity profile with a driver agent to which a result of learning is applied.
14. The system of claim 11 , wherein the agent implementation unit is configured to determine an APS/BPS variation based on the state variable and the reward variable and determines the APS/BPS command value based on the APS/BPS variation and the APS/BPS command value at a time point ahead of a predetermined time from a current time point.
15. The system of claim 11 , wherein the prediction value for the future velocity of the vehicle is determined based on past velocity information of the vehicle corresponding to at least two different time points.
16. The system of claim 11 , wherein the prediction value for the future velocity of the vehicle is determined based on i) a value obtained by passing past acceleration information of the vehicle through a low pass filter and ii) a current velocity of the vehicle.
17. The system of claim 11 , wherein the past APS/BPS command value is a past APS/BPS value during a predetermined period.
18. The system of claim 11 , wherein the prediction value for the future velocity comprises a prediction value for the future velocity during a predetermined period.
19. The system of claim 11 , wherein the state variable comprises at least one of a past velocity of the vehicle, an error of an actual velocity with respect to a past target velocity of the vehicle, or a target velocity of the vehicle.
20. The system of claim 19 , wherein the target velocity of the vehicle has a different value depending on a driving mode of the vehicle.