Battery Thermal Management System Control Strategy Based on Deep Reinforcement Learning

XI Chunfu; ZHAO Dongpeng; HUANG Chi; ZOU Zihao; HUANG Kunjie; XIE Yi

doi:null

您当前的位置：

首页 >

文章列表页 >

Battery Thermal Management System Control Strategy Based on Deep Reinforcement Learning

更新时间：2025-09-15

- Battery Thermal Management System Control Strategy Based on Deep Reinforcement Learning
- JOURNAL OF XI’AN JIAOTONG UNIVERSITY (2025)
- 作者机构：
  
  1.中国汽车工程研究院股份有限公司,重庆市重庆市401122
  2.天津大学机械工程学院,天津市天津市300350
  3.重庆大学低品位能源利用技术及系统教育部重点实验室,重庆市重庆市400044
  4.重庆大学能源与动力工程学院,重庆市重庆市400044
  5.重庆大学机械与运载工程学院,重庆市重庆市400044
- 作者简介：
- 基金信息：
- DOI：
  CLC： TM912
- Received：30 June 2025，
  
  Revised：2025-09-02，
  
  Accepted：15 September 2025，
- 稿件说明：
移动端阅览
XI Chunfu, ZHAO Dongpeng, HUANG Chi, et al. Battery Thermal Management System Control Strategy Based on Deep Reinforcement Learning[J/OL]. JOURNAL OF XI’AN JIAOTONG UNIVERSITY, 2025.
DOI：

XI Chunfu, ZHAO Dongpeng, HUANG Chi, et al. Battery Thermal Management System Control Strategy Based on Deep Reinforcement Learning[J/OL]. JOURNAL OF XI’AN JIAOTONG UNIVERSITY, 2025. DOI：

摘要

针对电动汽车电池热管理系统中传统控制方法温控精度不足及环境适应性差的难题，提出基于深度强化学习的智能控制方法。基于电池热电耦合模型与制冷空调系统模型，应用强化学习中的双延迟深度确定性策略（TD3）算法进行控制策略训练，通过双重价值网络与延迟策略更新机制，克服传统强化学习中的过高估计问题。训练结果表明：在夏季充电的训练工况下能够将电池包平均温度控制在25 ℃左右，在冬季充电的训练工况下能够将电池包平均温度控制在20 ℃左右，电池模组之间的最大温差控制在1 ℃以内，并且在控制动作上，智能体控制的压缩机转速的调整更为平缓；对比比例-积分-微分（PID）控制、开关控制，夏季放电时最高节能32.1%，充电时最高节能15.8%，冬季放电时最高节能17.0%，充电时最高节能26.3%。此外，在环境条件变化时，智能体能够及时调整控制动作，将电池包的温度控制在目标温度附近。该研究利用TD3强化学习算法能够在多变的环境条件下平稳、精准地控制电池热管理系统，证明了强化学习在电池热管理中的可行性与有效性。

Abstract

To address the challenges of insufficient temperature control accuracy and poor environmental adaptability in traditional control methods for electric vehicle battery thermal management systems

an intelligent control approach based on deep reinforcement learning is proposed. An environment is constructed by combining a battery electro-thermal coupling model with an air-conditioning system model

and the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm in reinforcement learning is applied for control strategy training. By incorporating dual critic networks and a delayed policy update mechanism

the algorithm effectively mitigates the overestimation issue commonly observed in conventional reinforcement learning methods. Training results demonstrate that

under summer charging conditions

the average battery pack temperature can be maintained at approximately 25 °C

while under winter charging conditions

it can be stabilized at around 20 °C

with the maximum temperature difference between modules controlled within 1 °C. Furthermore

the compressor speed adjustments executed by the agent are smoother. Compared with PID and ON/OFF control

the proposed method achieves up to 32.1% energy savings during summer discharging and 15.8% during summer charging

as well as up to 17.0% energy savings during winter discharging and 26.3% during winter charging. In addition

the agent adapts promptly to environmental variations

ensuring that the battery pack temperature remains close to the target. These findings demonstrate that the TD3-based reinforcement learning algorithm enables stable and precise control of battery thermal management systems under varying conditions

confirming the feasibility and effectiveness of reinforcement learning in this application.

关键词

Keywords

references

潘家华 , 董秀成 , 崔洪建等 . 欧洲能源危机及其影响分析 [J ] . 国际经济评论 , 2023 , ( 01 ): 9 - 37+4 .

PAN J H , DONG X C , CUI H J , et al . Analysis of European Energy Crisis and Its Impact [J ] . International Economic Review , 2023 , ( 01 ): 9 - 37+4 .

樊大磊 , 王宗礼 , 王彧嫣等 . 2024年国内外油气资源形势分析及展望 [J ] . 中国矿业 , 2025 , 34, ( 1 ): 46 - 54 .

FAN D L , WANG Z L , WANG Y Y , et al . Analysis of oil and gas resources situation at home and abroad in 2024 and outlook [J ] . China Mining Magazine , 2025 , 34, ( 1 ): 46 - 54 .

HUANG A S , ZHANG L , CHENG , W X , et al . CO2 emissions associated with China's real estate development: 2000-2020 [J ] . JOURNAL OF ENVIRONMENTAL SCIENCES , 2025 , 156 , 495 - 505 .

李晓易 , 谭晓雨 , 吴睿等 . 交通运输领域碳达峰、碳中和路径研究 [J ] . 中国工程科学 , 2021 , 23 ( 06 ): 15 - 21 .

LI X Y , TAN X Y , WU R , et al . Paths for Carbon Peak and Carbon Neutrality in Transport Sector in China [J ] . Strategic Study of CAE , 2021 , 23 ( 06 ): 15 - 21 .

乔英俊 , 赵世佳 , 伍晨波等 . “双碳”目标下我国汽车产业低碳发展战略研究 [J ] . 中国软科学 , 2022 , ( 06 ): 31 - 40 .

QIAO Y J , ZHAO S J , WU C B , et al . Research on Low-Carbon Development Strategy of China’s Automotive Industry with the “Carbon-Peak and Carbon-Neutrality” Goal [J ] . China Soft Science , 2022 , ( 06 ): 31 - 40 .

欧阳明高 . 能源革命与新能源智能汽车 [J ] . 中国工业和信息化 , 2019 , ( 11 ): 21 - 24 .

OUYANG M G . Energy Revolution and New Energy Intelligent Vehicles [J ] . China Industry & Information Technology , 2019 , ( 11 ): 21 - 24 .

孙叶 , 刘锴 . 里程焦虑对纯电动汽车使用意愿的影响 [J ] . 武汉理工大学学报（交通科学与工程版） , 2017 , 41, ( 1 ): 87 - 91 .

SUN Y , LIU K . Impact of Mileage Anxiety on Intention to Use Pure Electric Vehicle [J ] . Journal of Wuhan University of Technology(Transportation Science & Engineering) , 2017 , 41, ( 1 ): 87 - 91 .

于旭东 , 徐爽 , 李科迪 . 低温环境对纯电动汽车续驶里程的影响因素研究 [J ] . 汽车电器 , 2023 , ( 2 ): 1 - 3, 6 .

YU X D , XU S , LI K D . Study on Factors of Low-temperature Driving Range of Pure Electric Vehicle [J ] . Auto Electric Parts , 2023 , ( 2 ): 1 - 3, 6 .

ZHANG Z Q , LIU C C , CHEN X N , et al . Annual energy consumption of electric vehicle air conditioning in China [J ] . APPLIED THERMAL ENGINEERING , 2017 , 125 , 567 - 574 .

李哲 , 韩雪冰 , 卢兰光等 . 动力型磷酸铁锂电池的温度特性 [J ] . 机械工程学报 , 2011 , 47, ( 18 ): 115 - 120 .

LI Z , HAN X B , LU L G , et al . Temperature Characteristics of Power LiFePO4 Batteries [J ] . Journal of Mechanical Engineering , 2011 , 47, ( 18 ): 115 - 120 .

LU L G , HAN X B , LI J Q , et al . A review on the key issues for lithium-ion battery management in electric vehicles [J ] . JOURNAL OF POWER SOURCES , 2013 , 226 , 272 - 288 .

FENG X N , OUYANG M G , LIU X , et al . Thermal runaway mechanism of lithium-ion battery for electric vehicles: A review [J ] . ENERGY STORAGE MATERIALS , 2018 , 10 , 246 - 267 .

王从飞 , 曹锋 , 李明佳等 . 碳中和背景下新能源汽车热管理系统研究现状及发展趋势 [J ] . 科学通报 , 2021 , 66, ( 32 ): 4112 - 4128 .

WANG C F , CAO F , LI M J , et al . Research status and future development of thermal management system for new energy vehicles under the background of carbon neutrality [J ] . Chinese Science Bulletin , 2021 , 66, ( 32 ): 4112 - 4128 .

CHENG H Y , JUNG S H , KIM Y B . Battery thermal management system optimization using Deep reinforced learning algorithm [J ] . Applied Thermal Engineering , 2023 , 121759 -.

张蕾 , 杨洋 , 马菁等 . 液冷动力电池系统热管理控制策略优化探究 [J ] . 电源学报 , 2022 .

ZHANG L , YANG Y , MA Q , et al . Optimization of Thermal Management Control Strategy for Liquid Cooled Power Battery System [J ] . Journal of Power Supply , 2022 .

CEN J W , JIANG F M . Li-ion power battery temperature control by a battery thermal management and vehicle cabin air conditioning integrated system [J ] . Energy for Sustainable Development , 2020 , 57 : 141 - 148 .

WEI C , HOFMAN T , CAARLS E , et al . Zone Model Predictive Control for Battery Thermal Management including Battery Aging and Brake Energy Recovery in Electrified Powertrains [C ] . 9th IFAC International Symposium on Advances in Automotive Control (AAC) , 2019 : 303 - 308 .

HASKARA I , Hegde B , CHANG C . Reinforcement learning based EV energy management for integrated traction and cabin thermal management considering battery aging [J ] . IFAC-Papers Online , 2022 , 55, ( 24 ): 348 - 353 .

JOO S , LEE D , KIM M , et al . Multi-Agent Reinforcement Learning Based Actuator Control for EV HVAC Systems [J ] . IEEE ACCESS , 2023 , 11 , 7574 - 7587 .

XIE Y , LIU Z M , LI K N , et al . An improved intelligent model predictive controller for cooling system of electric vehicle [J ] . Applied Thermal Engineering , 2021 , 182 : 116084 .

KLOTE J , IM L , SCHICK J , et al . American Society of Heating, Refrigerating and Air-Conditioning Engineers Inc; proceedings of the ASHRAE Transactions: Symposia, F, 1995 [C ] .

GNIELINSKI V . New equations for heat and mass transfer in turbulent pipe and channel flow [J ] . International chemical engineering , 1976 , 16 ( 2 ): 359 - 67 .

CAVALLINI A , ZECCHIN R . A dimensionless correlation for heat transfer in forced convection condensation; proceedings of the International Heat Transfer Conference Digital Library, F, 1974 [C ] . Begel House Inc .

BERNARDI D , PAWLIKOWSKI E , NEWMAN J . A general energy balance for battery systems [J ] . 1984

封居强 , 司玉文 , 伍龙等 . 基于动态综合型等效电路模型的动力电池特性分析 [J ] . 储能科学与技术 , 2020 , 9, ( 3 ): 986 - 992 .

FENG J Q , SI Y W , WU L , et al . Analysis of dynamic battery characteristics based on dynamic synthesis equivalent circuit model [J ] . Energy Storage Science and Technology , 2020 , 9, ( 3 ): 986 - 992 .

JOHNSON V . Battery performance models in ADVISOR [J ] . Journal of power sources , 2002 , 110 ( 2 ): 321 - 9 .

杨鹏 . 基于模型预测控制的电动汽车整车热管理控制策略研究 [D ] . 重庆：重庆大学 , 2022 .

YANG P . Research on Thermal Management Control Strategy for Electric Vehicle Based on Model Predictive Control [D ] . Chongqing : Chongqing University , 2022 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Control Strategy for Battery Thermal Management System Using Deep Reinforcement Learning

Hierarchical Twin-Delayed Policy Gradient Reinforcement Learning for Intelligent Cooperative Control of Aircraft

Performance Analysis of Power Battery Cooling System under Different Transcritical CO2 Direct Cooling System Architectures

Function Mapping and Scheduling Method of Virtual Network Combining Genetic Algorithm and Reinforcement Learning

Research on Heat Transfer Characteristics and Structural Optimization of NewWaveform Microchannels for Lithium-Ion Batteries Cooling Plates

Related Author

XIE Yi

HUANG Chi

ZHAO Dongpeng

XI Chunfu

ZHANG Guanghua

MA Yu

AN Dou

LIN Xixiang

Related Institution

China Automotive Engineering Research Institute Co., Ltd.

School of Automation Science and Engineering, Xi'an Jiaotong University

School of Electronics and Control Engineering, Chang'an University

Yuxin Automotive Thermal Management Technology Co

National Innovation Platform (Centre)for Industry-Education Integration of Energy Storage Technology, Xi'an Jiaotong University

⁰