IP Library › Granted Patent US 12,325,322
Granted Patent B2
US 12,325,322 · App. 16/597,441 · Granted Jun 10, 2025

Physical model-guided machine learning framework for energy management of vehicles

Inventors: William Northrop (Minneapolis, MN); Andrew Kotz (Minneapolis, MN); PengYue Wang (Minneapolis, MN)
Assignee: Regents of the University of Minnesota
B60L58/12B60L50/60G06N3/04G06N3/08B60L2240/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,325,322
App. No.
16/597,441
Granted
Jun 10, 2025
Kind
B2
Abstract

A method of determining when to increase an amount electrical energy available to a vehicle includes setting a parameter for a function describing a reference state of charge as a function of distance traveled, wherein the reference state of charge represents a state of charge of the vehicle at which the amount of electrical energy available to the vehicle should be increased. For each trip of the vehicle, the parameter for the function is modified so that different trips of a same vehicle use different functions for the reference state of charge.

Claims (37)

1. A method of increasing an amount of electrical energy available to a vehicle, the method comprising:

setting at least one parameter for a function describing a reference state of charge as a function of distance traveled, wherein the reference state of charge represents a state of charge of the vehicle at which the amount of electrical energy available to the vehicle should be increased, wherein setting the at least one parameter causes different trips of a same vehicle to use different functions for the reference state of charge; and

during a trip:

using the at least one parameter set for the function and a distance traveled during the trip to determine a value for the reference state of charge; and

determining that a current state of charge of the vehicle is equal to or less than the reference state of charge and, in response, instructing an internal combustion engine to start or sending a notification to recharge the vehicle using an external charger.

2. The method of claim 1 , wherein the setting the at least one parameter comprises setting the at least one parameter before a trip to form at least one modified parameter and using the at least one modified parameter for an entirety of the trip.

3. The method of claim 2 , wherein the setting the at least one parameter before starting a trip comprises setting the at least one parameter based on a prior model created at least in part from past trips of the same vehicle.

4. The method of claim 3 , wherein the prior model is created at least in part from a respective best value for each of the at least one parameter determined for a most recent past trip of the same vehicle, wherein the respective best value for each of the at least one parameter results in the least amount of electrical energy being made available to the vehicle while preventing the state of charge from crossing below a threshold during the most recent past trip.

5. The method of claim 4 , wherein the best value is determined using a vehicle model to estimate changes in the state of charge for the most recent past trip.

6. The method of claim 1 , wherein the setting the at least one parameter comprises modifying the at least one parameter during the trip.

7. The method of claim 6 , wherein the setting the at least one parameter comprises using a neural network to select a change in the at least one parameter based on a state of the vehicle.

8. The method of claim 7 , wherein the neural network is trained using reinforcement learning.

9. The method of claim 1 , wherein the vehicle is a range extended hybrid electric vehicle.

10. The method of claim 9 , wherein the vehicle is an all-electric vehicle.

11. A computer system comprising:

a communication interface receiving trip information from a vehicle for a time period;

a processor, receiving the trip information from the communication interface and performing steps comprising:

using the trip information for the time period to change how a reference state of charge is determined from a distance traveled during a trip, wherein the reference state of charge represents a state of charge of the vehicle at which an amount of electrical energy available to the vehicle should be increased; and

providing to a processor in a vehicle an indication of how the reference state of charge is to be determined from the distance traveled so as to cause the processor to:

determine the reference state of charge from the distance traveled during the trip; and

determine that a current state of charge of the vehicle is equal to or less than the reference state of charge and, in response, instructing an internal combustion engine to start or sending a notification to recharge the vehicle using an external charger.

12. The computer system of claim 11 , wherein the time period covers the entirety of a previous trip.

13. The computer system of claim 12 , wherein changing how the reference state of charge is determined comprises changing how the reference state of charge is determined based on a prior model of a parameter used to determine the reference state of charge.

14. The computer system of claim 13 , wherein the prior model is created at least in part from a best value for the parameter determined for a most recent past trip of the vehicle, wherein the best value results in the least amount of electrical energy being made available to the vehicle while preventing the state of charge from crossing below a threshold during the most recent past trip.

15. The computer system of claim 14 , wherein the best value is determined using a vehicle model to estimate changes in the state of charge for the last trip.

16. The computer system of claim 11 , wherein the time period covers less than all of a trip in progress.

17. The computer system of claim 16 , wherein using the trip information for the time period to change how the reference state of charge is determined comprises using the trip information to identify a state and applying the state to a neural network to obtain the change in how the reference state of charge is determined.

18. A computing device comprising:

a memory storing trip information for a vehicle;

a processor executing instructions to perform steps comprising:

using at least some of the trip information to alter a function used to determine a reference state of charge from a distance traveled during a trip, wherein the reference state of charge represents a state of charge of the vehicle at which an amount of electrical energy available to the vehicle should be increased and wherein the reference state of charge changes during a vehicle trip;

using the altered function to determine the reference state of charge from the distance traveled;

determining that a current state of charge of the vehicle is equal to or less than the reference state of charge and, in response, instructing an internal combustion engine to start or sending a notification to recharge the vehicle using an external charger.

19. The computing device of claim 18 , wherein the trip information used to alter the function comprises trip information for an entirety of a latest trip.

20. The computing device of claim 19 , wherein the trip information is used to alter a prior probability distribution and the prior probability distribution is used to alter the function.

21. The computing device of claim 18 , wherein the trip information comprises trip information for a current trip.

22. The computing device of claim 21 , wherein altering the function comprises determining a state from the trip information and applying the state to a neural network to determine how to alter the function.

Assignments (3)
CONFIRMATORY LICENSE Recorded Feb 12, 2024
From: REGENTS OF THE UNIVERSITY OF MINNESOTA
To: US DEPARTMENT OF ENERGY
Reel/Frame 066438/0185 →
CONFIRMATORY LICENSE Recorded Jul 15, 2022
From: REGENTS OF THE UNIVERSITY OF MINNESOTA
To: U.S. DEPARTMENT OF ENERGY
Reel/Frame 060526/0657 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 4, 2021
From: KOTZ, ANDREW; NORTHROP, WILLIAM; WANG, PENGYUE
To: REGENTS OF THE UNIVERSITY OF MINNESOTA
Reel/Frame 055146/0132 →
Continuity (2)
Provisional Application 62743321 · Oct 9, 2018
Related Publication 20200108732A1 · Apr 9, 2020
References Cited (78)
US 20050228553A1 · Tryon · 2005 [cited by examiner]
US 20120290159A1 · McGee · 2012 [cited by examiner]
US 20160200314A1 · Tagami · 2016 [cited by examiner]
US 20160355172A1 · Park · 2016 [cited by examiner]
Rajagopalan et al., Intelligent Control of Hybrid Electric Vehicles Using GPS Information, SAE Technical Paper, No. 724, 11 pages, 2002. [cited by applicant]
Ripaccioli et al., A stochastic model predictive control approach for series hybrid electric vehicle power management, in Proceeding American Control Conference (ACC), pp. 5844-5849, 2010. [cited by applicant]
Rousseau et al., Plug-in hybrid electric vehicle control strategy parameter optimization, presented at the Electric Vehical Symp.-23, 14 pages, 2007. [cited by applicant]
Rowland et al., An analysis of categorical distributional reinforcement learning, Proceedings of the 21st Conference on Artificial Intelligence and Statistics (AISTATS), 9 pages, 2018. [cited by applicant]
Salmasi, Control Strategies for Hybrid Electric Vehicles: Evolution, Classification, Comparison, Fol. 56, No. 5, pp. 2393-2404, 2007. [cited by applicant]
Serrao et al., A Comparative Analysis of Energy Management Strategies for Hybrid Electric Vehicles, Journal of Dynamic Systems, Measurement, and Control, vol. 133, No. 3, pp. 031012-1-031012-9, 2011. [cited by applicant]
Sharer et al., Plug-in hybrid electric vehicle control strategy: Comparison between EV and charge-depleting options, SAE International, Paper No. 2008-01-0460, 2008. [cited by applicant]
Shekhar et al., Encyclopedia of GIS. Springer Publishing Company, 36 pages, 2007. [cited by applicant]
Silver et al., Deterministic policy gradient algorithms, Proceedings of the 31st International Conference on Machine Learning, vol. 32. pp. 387-395, 2014. [cited by applicant]
Stockar et al., Energy-Optimal Control of Plug-in Hybrid Electric Vehicles for Real-World Driving Cycles, IEEE Transactions on Vehicular Technology, vol. 60, No. 7, pp. 2949-2962, 2011. [cited by applicant]
Sun et al., Dynamic Traffic Feedback Data Enabled Energy Management in Plug-in Hybrid Electric Vehicles, IEEE Transactions on Control Systems Technology, vol. 23, No. 3, pp. 1075-1086, 2015. [cited by applicant]
Taherzadeh et al., A New Efficient Fuel Optimization in Blended Charge Depletion/Charge Sustenance Control Strategy for Plug-In Hybrid Electric Vehicles, IEEE Transactions on Intelligent Vehicles, vol. 3, No. 3, pp. 374… [cited by applicant]
Tian et al., Adaptive Fuzzy Logic Energy Management Strategy Based on Reasonable SOC Reference Curve for Online Control of Plug-in Hybrid Electric City Bus, IEEE Transactions on Intelligent Transportation Systems, vol. … [cited by applicant]
Tianheng et al., A Supervisory Control Strategy for Plug-In Hybrid Electric Vehicles Based on Energy Demand Prediction and Route Preview, IEEE Transactions on Vehicular Technology, vol. 64, No. 5, pp. 1691-1700, 2015. [cited by applicant]
Tulpule et al., Effects of different PHEV control strategies on vehicle performance, Proceedings of the American Control Conference, pp. 3950-3955, 2009. [cited by applicant]
Unal et al., Quantification of highway vehicle emissions hot spots based upon on-board measurements., Journal of the Air and Waste Management Association, vol. 54, No. 2, pp. 130-140, 2004. [cited by applicant]
Vinitsky et al., Lagrangian Control through Deep-RL: Applications to Bottleneck Decongestion, 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 759-765, 2018. [cited by applicant]
Wang et al., Adaptive RBF network for parameter estimation and stable air-fuel ratio control, Neural Networks, vol. 21, No. 1, pp. 102-112, 2008. [cited by applicant]
Wang et al., Deep Reinforcement Learning with Knowledge Transfer for Online Rides Order Dispatching, IEEE International Conference on Data Mining (ICDM), pp. 617-626, 2018. [cited by applicant]
Wang et al., A Deep Reinforcement Learning Framework for Energy Management of Range-Extended Hybrid Electric Delivery Vehicles, IEEE Intelligent Vehicles Symposium, pp. 1837-1842, 2019. [cited by applicant]
Wirasingha et al., Classification and Review of Control Strategies for Plug-In Hybrid Electric Vehicles, IEEE Transactions on Vehicular Technology, vol. 60, No. 1, pp. 111-122, 2011. [cited by applicant]
Wu et al., Development and Evaluation of an Intelligent Energy-Management Strategy for Plug-in Hybrid Electric Vehicles, IEEE Transactions on Intelligent Transportation Systems, vol. 15, No. 3, pp. 1091-1100, 2014. [cited by applicant]
Xie et al., Transdisciplinary Foundations of Geospatial Data Science, International Journal of Geo-Information, ISPRS, vol. 6, No. 12, 24 pages, 2017. [cited by applicant]
Xie et al., Time-Efficient Stochastic Model Predictive Energy Management for a Plug-In Hybrid Electric Bus with Adaptive Reference State-of-Charge Advisory, IEEE Transactions on Vehicular Technology, vol. 67, No. 7, pp.… [cited by applicant]
Yoo et al., System Integration and Power-Flow Management for a Series Hybrid Electric Vehicle Using Supercapacitors and Batteries, IEEE Transactions on Industry Applications, vol. 44, No. 1, pp. 108-114, 2008. [cited by applicant]
Zeng et al., A Parallel Hybrid Electric Vehicle Energy Management Strategy Using Stochastic Model Predictive Control With Road Grade Preview, IEEE Transactions on Control Systems Technology, vol. 23, No. 6, pp. 2416-242… [cited by applicant]
Zeng et al., Optimizing the Energy Management Strategy for Plug-In Hybrid Electric Vehicles With Multiple Frequent Routes, IEEE Transactions on Control Systems Technology, vol. 27, No. 1, pp. 394-400, 2017. [cited by applicant]
Zhang et al., Role of Terrain Preview in Energy Management of Hybrid Electric Vehicles, IEEE Transactions on Vehicular Technology, vol. 59, No. 3, pp. 1139-1147, 2010. [cited by applicant]
Zhang et al., Real-Time Energy Management Strategy Based on Velocity Forecasts Using V2V and V2I Communications, IEEE Transactions on Intelligent Transportation Systems, vol. 18, No. 2, pp. 416-430, 2017. [cited by applicant]
Zhang et al., Predictive Energy Management Strategy for Fully Electric Vehicles Based on Preceding Vehicle Movement, IEEE Transactions on Intelligent Transportation Systems, vol. 18, No. 11, pp. 3049-3060, 2017. [cited by applicant]
Ali et al., Discovering Non-compliant Window Co-Occurance Patterns: A Summary of Results, Advances in Spatial and Temporal Databases, pp. 391-410, 2015. [cited by applicant]
Ali et al., Future Connected Vehicles: Challenges and Opportunities for Spatio-temporal Computing, ACM Int. Conf. Adv. Geogr. Inf. Syst., Article No. 14, pp. 20-23, 2015. [cited by applicant]
Ali et al., Emerging Applications of Spatial Network Big Data in Transportation, Big Data and Computational Intelligence in Networking, Chapter 18, pp. 433-540, 2017. [cited by applicant]
Ambuhl et al., Predictive reference signal generator for hybrid electric vehicles, IEEE Transactions of Vehicular Technology, vol. 58, No. 9, pp. 4730-4740, 2009. [cited by applicant]
Badshah et al., Particle Emissions from Light-Duty Vehicles during Cold-Cold Start, SAE International Journal of Engines, vol. 9, No. 3, 2016. [cited by applicant]
Bellemare et al., A distributional perspective on reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning (ICML), 19 pages, 2017. [cited by applicant]
Bevly et al., Integrating INS Sensors with GPS Velocity Measurements for Continuous Estimation of Vehicle Sideslip and Tire Cornering Stiffness, in Proceedings of the American Control Conference, vol. 27, No. 6, pp. 25-… [cited by applicant]
Bojarski et al., End to end learning for self-driving cars, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 9 pages, 2016. [cited by applicant]
Brahma et al., Optimal energy management in series hybrid electric vehicles, Proceedings of the American Control Conference, vol. 1, pp. 60-64, 2000. [cited by applicant]
Chen et al., Temporal-Difference Learning-Based Stochastic Energy Management for Plug-in Hybrid Electric Buses, in IEEE Transactions on Intelligent Transportation Systems, vol. 20, No. 6, pp. 2378-2388, 2019. [cited by applicant]
Dabney et al., Implicit quantile networks for distributional reinforcement learning, Proceedings of the 35th International Conference on Machine Learning, 14 pages, 2018. [cited by applicant]
Daily et al., The use of GPS for vehicle stability control systems, IEEE Transactions on Industrial Electronics, vol. 51, No. 2, pp. 270-277, 2004. [cited by applicant]
Davenport et al. Big Data in Big Companies, International Institute for Analytics, pp. 1-31, 2013. [cited by applicant]
Denis et al., Power Split Strategy Optimization of a Plug-in Parallel Hybrid Electric Vehicle, IEEE Transactions on Vehicular Technology, vol. 67, No. 1, pp. 315-326, 2018. [cited by applicant]
Di Cairano et al., Stochastic MPC With Learning for Driver-Predictive Vehicle Control and its Application to HEV Energy Management, IEEE Transactions on Control Systems Technology, vol. 22, No. 3, pp. 1018-1031, 2014. [cited by applicant]
Freyermuth et al., Comparison of Powertrain Configuration for Plug-in HEVs from a Fuel Economy Perspective, SAE International Journal of Engines, vol. 1, No. 1, pp. 392-398, 2008. [cited by applicant]
Fridholm et al., Robustness Comparison of Battery State of Charge Observers for Automotive Applications. Proceedings of the 19th World Congress IFAC, pp. 2138-2146, 2014. [cited by applicant]
Fridholm et al., Robust recursive impedance estimation for automotive lithium-ion batteries. Journal of Power Sources, vol. 304, 33-41, 2016. [cited by applicant]
Fridholm et al., Estimating power capability of aged lithium-ion batteries in presence of communication delays, Journal of Power Sources, No. 383, 24-33, 2018. [cited by applicant]
Gonder et al. Measuring and Reporting Fuel Economy of Plug-In Hybrid Electric Vehicles, National Renewable Energy aboratory, 15 pages, 2006. [cited by applicant]
Gouge et al., The spatial distribution of diesel transit bus emissions and people: implications of scale on exposure and impact assessment, Environmental Science Technology, vol. 44, No. 18, pp. 7163-7168, 2010. [cited by applicant]
Gunturi et al., Lagrangian Xgraphs: A Logical Data-Model for Spatio-Temporal Network Data: A Summary, Adv. Concept. Model., vol. 8823, pp. 201-211, 2014. [cited by applicant]
Haghani et al., A dynamic vehicle routing problem with time-dependent travel times, Computers and Operations Research, vol. 32, No. 11, pp. 2959-2986, 2005. [cited by applicant]
Hahn et al., GPS-based real-time identification of tireroad friction coefficient, IEEE Transactions on Control Systems Technology, vol. 10, No. 3, pp. 331-343, 2002. [cited by applicant]
Huang et al., Parameter optimization of power control strategy for series hybrid electric vehicle, in Proc. IEEE CEC, pp. 1989-1994, 2006. [cited by applicant]
Hubaux et al., The security and privacy of smart vehicles, IEEE Security and Privacy Mag., vol. 2, No. 3, pp. 49-55, 2004. [cited by applicant]
Johannesson et al., Assessing the Potential of Predictive Control for Hybrid Vehicle Powertrains Using Stochastic Dynamic Programming, IEEE Transactions on Intelligent Transportation Systems, vol. 8, No. 1, pp. 71-83, 2… [cited by applicant]
Katrasnik et al., Analysis of Energy Conversion Efficiency in Parallel and Series Hybrid Powertrains, IEEE Transactions on Vehicular Technology, vol. 56, No. 6, pp. 3649-3659, 2007. [cited by applicant]
Kermani et al., PHIL Implementation of Energy Management Optimization for a Parallel HEV on a Predefined Route, IEEE Transactions on Vehicular Technology, vol. 60, No. 3, pp. 782-792, 2011. [cited by applicant]
Kotz et al., Lagrangian Hotspots of In-Use NOX Emissions from Transit Buses, Environmental Science and Technology, vol. 50, 5750-5756, 2016. [cited by applicant]
Li et al., Physics-guided Energy-efficient Path Selection: A Summary of Results. 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, 10 pages, 2018. [cited by applicant]
Liang et al., A Deep Reinforcement Learning Network for Traffic Light Cycle Control, IEEE Transactions on Vehicular Technology, vol. 68, No. 2, pp. 1243-1253, 2019. [cited by applicant]
Lillicrap et al., Continuous control with deep reinforcement learning, Published as a conference paper at ICLR, 14 pages, 2016. [cited by applicant]
Lin et al., Machine learning-based energy management in a hybrid electric vehicle to minimize total operating cost, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 627-634, 2015. [cited by applicant]
Liu et al., Adaptive critic learning techniques for engine torque and air-fuel ratio control, IEEE Transactions on Systems, Man, Cybernetics—Part B Cybernetics, vol. 38, No. 4, pp. 988-993, 2008. [cited by applicant]
Liu et al., Power management for plug-in hybrid electric vehicles using reinforcement learning with trip information, in Proc. IEEE ITEC, pp. 1-6, 2014. [cited by applicant]
Liu et al., Impact of Time Varying Passenger Loading on Conventional and Electrified Transit Bus Energy Consumption, Transportation Research Board Annual Meeting, 9 pages, 2019. [cited by applicant]
Martinez et al., Energy Management in Plug-in Hybrid Electric Vehicles: Recent Progress and a Connected Vehicles Perspective, IEEE Transactions on Vehicular Technology, vol. 66, No. 6, pp. 4534-4549, 2017. [cited by applicant]
Martinez et al., Driving Style Recognition for Intelligent Vehicle Control and Advanced Driver Assistance: A Survey, IEEE Transactions on Intelligent Transportation Systems, vol. 19, No. 3, pp. 666-676, 2018. [cited by applicant]
Mnih et al., Human-level control through deep reinforcement learning. Nature, vol. 518, No. 7540, pp. 529-533, 2015. [cited by applicant]
Murphy, Conjugate Bayesian analysis of the Gaussian distribution, Univ. British Columbia, Canada, Technical Report, 29 pages, 2007. [cited by applicant]
Overington et al., High-Efficiency Control of Internal Combustion Engines in Blended Charge Depletion/Charge Sustenance Strategies for Plug-In Hybrid Electric Vehicles, in IEEE Transactions on Vehicular Technology, vol.… [cited by applicant]
Qi et al., Deep reinforcement learning-based vehicle energy efficiency autonomous learning system, IEEE Intelligent Vehicles Symposium (IV), pp. 1228-1233, 2017. [cited by applicant]
Qi et al., Development and Evaluation of an Evolutionary Algorithm-Based OnLine Energy Management System for Plug-In Hybrid Electric Vehicles, IEEE Transactions on Intelligent Transportation Systems, vol. 18, No. 8, pp.… [cited by applicant]