Vehicle control data generation method, vehicle controller, vehicle control system, and vehicle learning device
A vehicle control data generation method includes causing processing circuitry to execute an obtaining process that obtains a state of a vehicle and a specifying variable, an operating process that operates an electronic device, a reward calculating process that provides a greater reward when a characteristic of the vehicle meets a standard than when the characteristic does not meet the standard, and an updating process that updates relationship defining data. The update map outputs the updated relationship defining data. The reward calculating process includes a changing process that changes the reward, provided when the characteristic of the vehicle is a predetermined characteristic, such that the reward in a case where torque generated by an internal combustion engine is used to generate the propelling force of the vehicle differs from the reward in a case where the torque is not used to generate the propelling force.
1. A vehicle control data generation method, wherein a memory device stores relationship defining data that defines a relationship between a state of a vehicle including a rotating electric machine and an internal combustion engine and an action variable related to operation of an electronic device in the vehicle,
the generation method comprising causing processing circuitry to execute:
an obtaining process that obtains a specifying variable, the specifying variable specifying whether the state of the vehicle obtained based on a detection value of a sensor and torque generated by the internal combustion engine in a running state of the internal combustion engine are used to generate a propelling force of the vehicle;
an operating process that operates the electronic device;
a reward calculating process that provides, based on the state of the vehicle obtained by the obtaining process, a greater reward when a characteristic of the vehicle meets a standard than when the characteristic of the vehicle does not meet the standard; and
an updating process that updates the relationship defining data by inputting, to a predetermined update map, the state of the vehicle obtained by the obtaining process, a value of the action variable used to operate the electronic device, and the reward corresponding to the operation of the electronic device,
the update map outputs the updated relationship defining data so as to increase an expected return for the reward in a case where the electronic device is operated in accordance with the relationship defining data, and
the reward calculating process includes a changing process that changes the reward, provided when the characteristic of the vehicle is a predetermined characteristic, such that the reward in a case where the torque generated by the internal combustion engine in the running state of the internal combustion engine is used to generate the propelling force of the vehicle differs from the reward in a case where the torque is not used to generate the propelling force.
2. The vehicle control data generation method according to claim 1 , wherein
the reward calculating process includes a process that provides a greater reward when an energy use efficiency is high than when the energy use efficiency is low, and
the changing process includes a process that changes the reward such that increasing the energy use efficiency becomes more advantageous to obtain a great reward in the case where the torque generated by the internal combustion engine in the running state of the internal combustion engine is not used to generate the propelling force of the vehicle than in the case where the torque is used to generate the propelling force.
3. The vehicle control data generation method according to claim 1 , further comprising causing the processing circuitry to execute a process that generates control map data, based on the relationship defining data updated by the updating process, by associating the state of the vehicle with the value of the action variable that maximizes the expected return, the control map data using the state of the vehicle as an input and outputting the value of the action variable that maximizes the expected return.
4. A controller for a vehicle including a rotating electric machine and an internal combustion engine, the controller comprising:
a memory device configured to store relationship defining data that defines a relationship between a state of the vehicle and an action variable related to operation of an electronic device in the vehicle; and
processing circuitry, wherein
the processing circuitry is configured to execute:
an obtaining process that obtains a specifying variable, the specifying variable specifying whether the state of the vehicle obtained based on a detection value of a sensor and torque generated by the internal combustion engine in a running state of the internal combustion engine are used to generate a propelling force of the vehicle;
an operating process that operates, based on the relationship defining data, the electronic device in correspondence with a value of the action variable corresponding to the state of vehicle;
a reward calculating process that provides, based on the state of the vehicle obtained by the obtaining process, a greater reward when a characteristic of the vehicle meets a standard than when the characteristic of the vehicle does not meet the standard; and
an updating process that updates the relationship defining data by inputting, to a predetermined update map, the state of the vehicle obtained by the obtaining process, a value of the action variable used to operate the electronic device, and the reward corresponding to the operation of the electronic device,
the update map outputs the updated relationship defining data so as to increase an expected return for the reward in a case where the electronic device is operated in accordance with the relationship defining data, and
the reward calculating process includes a changing process that changes the reward, provided when the characteristic of the vehicle is a predetermined characteristic, such that the reward in a case where the torque generated by the internal combustion engine in the running state of the internal combustion engine is used to generate the propelling force of the vehicle differs from the reward in a case where the torque is not used to generate the propelling force.
5. A control system for a vehicle including a rotating electric machine and an internal combustion engine, the control system comprising:
a memory device configured to store relationship defining data that defines a relationship between a state of the vehicle and an action variable related to operation of an electronic device in the vehicle; and
processing circuitry, wherein
the processing circuitry is configured to execute:
an obtaining process that obtains a specifying variable, the specifying variable specifying whether the state of the vehicle obtained based on a detection value of a sensor and torque generated by the internal combustion engine in a running state of the internal combustion engine are used to generate a propelling force of the vehicle;
an operating process that operates, based on the relationship defining data, the electronic device in correspondence with a value of the action variable corresponding to the state of vehicle;
a reward calculating process that provides, based on the state of the vehicle obtained by the obtaining process, a greater reward when a characteristic of the vehicle meets a standard than when the characteristic of the vehicle does not meet the standard; and
an updating process that updates the relationship defining data by inputting, to a predetermined update map, the state of the vehicle obtained by the obtaining process, a value of the action variable used to operate the electronic device, and the reward corresponding to the operation of the electronic device,
the update map outputs the updated relationship defining data so as to increase an expected return for the reward in a case where the electronic device is operated in accordance with the relationship defining data,
the reward calculating process includes a changing process that changes the reward, provided when the characteristic of the vehicle is a predetermined characteristic, such that the reward in a case where the torque generated by the internal combustion engine in the running state of the internal combustion engine is used to generate the propelling force of the vehicle differs from the reward in a case where the torque is not used to generate the propelling force,
the processing circuitry includes a first processing circuitry mounted on the vehicle and a second processing circuitry that differs from an in-vehicle device,
the first processing circuitry is configured to execute at least the obtaining process and the operating process, and
the second processing circuitry is configured to execute at least the updating process.
6. A vehicle controller, comprising the first processing circuitry of the vehicle control system according to claim 5 .
7. A vehicle learning device, comprising the second processing circuitry of the vehicle control system according to claim 5 .