Reinforcement Learning-Based Energy Management Strategy Considering Driving Style for Hybrid Electric Vehicle

SHI Dehua; YUAN Chao; WANG Shaohua; ZHOU Weiqi; CHEN Long

doi:10.7652/xjtuxb202410005

您当前的位置：

首页 >

文章列表页 >

Reinforcement Learning-Based Energy Management Strategy Considering Driving Style for Hybrid Electric Vehicle

更新时间：2025-07-28

- Reinforcement Learning-Based Energy Management Strategy Considering Driving Style for Hybrid Electric Vehicle
- Vol. 58, Issue 10, Pages: 51-62(2024)
- 作者机构：
  
  1. 江苏大学汽车工程研究院,江苏,镇江,212013
  2. 江苏省新能源汽车电驱动系统与智能控制工程研究中心,江苏,镇江,212013
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202410005
  CLC： U463.2
- Online First：10 October 2024，
  
  Published：2024
- 稿件说明：
移动端阅览
SHI Dehua, YUAN Chao, WANG Shaohua, et al. Reinforcement Learning-Based Energy Management Strategy Considering Driving Style for Hybrid Electric Vehicle[J]. 2024, 58(10): 51-62.
DOI：

SHI Dehua, YUAN Chao, WANG Shaohua, et al. Reinforcement Learning-Based Energy Management Strategy Considering Driving Style for Hybrid Electric Vehicle[J]. 2024, 58(10): 51-62. DOI： 10.7652/xjtuxb202410005.

摘要

为了提升混合动力汽车能量管理策略对不同风格驾驶员的适应性

基于深度强化学习和等效燃油消耗最小策略(equivalent consumption minimization strategy

ECMS)

提出一种考虑驾驶风格的混合动力汽车能量管理策略。通过实车试验采集驾驶员行驶数据

基于采集数据进行驾驶员驾驶风格的聚类分析

建立驾驶风格识别模型; 构建基于强化学习和ECMS的能量管理策略

将驾驶风格系数作为强化学习状态变量

利用多种驾驶风格的组合工况训练深度确定性策略梯度智能体

获取不同工况和驾驶风格下ECMS等效因子

采用ECMS求解最优发动机、电机转矩分配以及变速箱挡位; 搭建硬件在环测试平台

并基于实际采集的不同驾驶员驾驶数据构建测试工况

验证所提出控制策略的有效性。研究结果表明

相较于基于规则策略、基于等效因子比例修正的自适应ECMS以及DRL-SAC策略

提出的考虑驾驶风格的强化学习能量管理策略使整车能量消耗分别降低16.35%、11.11%和7.56%

所提控制策略的有效性得到了验证。

Abstract

A considering driving style energy management strategy is proposed to enhance the adaptability of energy management strategies for different driving styles in hybrid electric vehicles. The strategy combines deep reinforcement learning with the equivalent consumption minimization strategy(ECMS). Real vehicle experiments are conducted to collect driving data

which is then subjected to clustering analysis to identify distinct driving styles. A driving style recognition model is developed based on this data. The energy management strategy is built using reinforcement learning and ECMS

with driving style coefficients serving as the reinforcement learning state variables. A deep deterministic policy gradient agent is trained using various combinations of driving styles and operating conditions to determine ECMS equivalent factors for different driving styles and conditions. The ECMS is employed to optimize the engine

motor torque allocation

and gearbox gear selection. To validate the effectiveness of the proposed control strategy

a hardware in the loop testing platform is constructed

and test scenarios are generated using real driving data from different drivers. The research findings demonstrate that the reinforcement learning-based energy management strategy considering driving style reduces overall vehicle energy consumption by 16.35%

11.11%

and 7.56% compared with rule-based strategy

equivalent factor ratio correction-based adaptive ECMS

and DRL-SAC strategy

respectively. The effectiveness of the proposed control strategy is successfully validated.

关键词

Keywords

references

王跃飞, 王志, 孙睿, 等. 基于驾驶意图多步预测的智能网联HEV等效排放最小控制策略 [J]. 机械工程学报, 2023, 59(18): 271-282.

WANG Yuefei, WANG Zhi, SUN Rui, et al. Equivalent emission minimization strategy of intelligent connected HEV based on multi-step prediction of driving intention [J]. Journal of Mechanical Engineering, 2023, 59(18): 271-282.

黄康, 王强, 邱明明, 等. 考虑模式切换频率的多模式混合动力汽车参数优化 [J]. 西安交通大学学报, 2019, 53(7): 99-107.

HUANG Kang, WANG Qiang, QIU Mingming, et al. Parameter optimization of multi-mode hybrid vehicle considering mode-switching frequency [J]. Journal of Xi'an Jiaotong University, 2019, 53(7): 99-107.

YANG Chao, DU Xuelong, WANG Weida, et al. Variable optimization domain-based cooperative energy management strategy for connected plug-in hybrid electric vehicles [J]. Energy, 2024, 290: 130206.

黄硕, 李亮, 杨超, 等. 基于规则修正的同轴并联混合动力客车瞬时优化能量分配策略 [J]. 机械工程学报, 2014, 50(20): 113-121.

HUANG Shuo, LI Liang, YANG Chao, et al. Rule correction-based instantaneous optimal energy management strategy for single-shaft parallel hybrid electric bus [J]. Journal of Mechanical Engineering, 2014, 50(20): 113-121.

DU Guodong, ZOU Yuan, ZHANG Xudong, et al. Deep reinforcement learning based energy management for a hybrid electric vehicle [J]. Energy, 2020, 201: 117591.

王文彬, 田韶鹏, 郑青星, 等. 基于FA的等效燃油消耗最小控制策略优化 [J]. 江苏大学学报(自然科学版), 2022, 43(2): 147-153.

WANG Wenbin, TIAN Shaopeng, ZHENG Qingxing, et al. Optimization of equivalent fuel consumption minimization strategy based on firefly algorithm [J]. Journal of Jiangsu University(Natural Science Edition), 2022, 43(2): 147-153.

CHEN Yao, WANG Ke, LU J J. Feature selection for driving style and skill clustering using naturalistic driving data and driving behavior questionnaire [J]. Accident Analysis Prevention, 2023, 185: 107022.

ZHANG Zhen, ZHANG Tiezhu, HONG Jichao, et al. Energy management strategy of a novel electric-hydraulic hybrid vehicle based on driving style recognition [J]. Sustainable Energy Fuels, 2023, 7(2): 420-430.

TIAN Xiang, CAI Yingfeng, SUN Xiaodong, et al. Incorporating driving style recognition into MPC for energy management of plug-in hybrid electric buses [J]. IEEE Transactions on Transportation Electrification, 2023, 9(1): 169-181.

GUO Qiuyi, ZHAO Zhiguo, SHEN Peihong, et al. Adaptive optimal control based on driving style recognition for plug-in hybrid electric vehicle [J]. Energy, 2019, 186: 115824.

ZHU Zhaoxuan, LIU Yuxing, CANOVA M. Energy management of hybrid electric vehicles via deep Q-networks [C]//2020 American Control Conference(ACC). Piscataway, NJ, USA: IEEE, 2020: 3077-3082.

LEE W, JEOUNG H, PARK D, et al. A real-time intelligent energy management strategy for hybrid electric vehicles using reinforcement learning [J]. IEEE Access, 2021, 9: 72759-72768.

XU Bin, HOU Jun, SHI Junzhe, et al. Learning time reduction using warm-start methods for a reinforcement learning-based supervisory control in hybrid electric vehicle applications [J]. IEEE Transactions on Transportation Electrification, 2021, 7(2): 626-635.

HU Dong, ZHANG Yuanyuan. Deep reinforcement learning based on driver experience embedding for energy management strategies in hybrid electric vehicles [J]. Energy Technology, 2022, 10(6): 2200123.

WANG Hanchen, YE Yiming, ZHANG Jiangfeng, et al. A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle [J]. Energy, 2023, 266: 126497.

HE Kun, QIN Dongchen, CHEN Jiangyi, et al. Adaptive equivalent consumption minimization strategy for fuel cell buses based on driving style recognition [J]. Sustainability, 2023, 15(10): 7781.

YANG Sen, WANG Wenshuo, ZHANG Fengqi, et al. Driving-style-oriented adaptive equivalent consumption minimization strategies for HEVs [J]. IEEE Transactions on Vehicular Technology, 2018, 67(10): 9249-9261.

GONG Changchao, HU Minghui, LI Shuxian, et al. Equivalent consumption minimization strategy of hybrid electric vehicle considering the impact of driving style [J]. Proceedings of the Institution of Mechanical Engineers: Part D Journal of Automobile Engineering, 2019, 233(10): 2610-2623.

王旭, 马菲, 廖小棱, 等. 基于多分类监督学习的驾驶风格特征指标筛选 [J]. 交通信息与安全, 2022, 40(1): 162-168.

WANG Xu, MA Fei, LIAO Xiaoleng, et al. Feature selection for recognition of driving styles based on multi-classification and supervised learning [J]. Journal of Transport Information and Safety, 2022, 40(1): 162-168.

李经纬, 赵治国, 沈沛鸿, 等. 驾驶风格K-means聚类与识别方法研究 [J]. 汽车技术, 2018(12): 8-12.

LI Jingwei, ZHAO Zhiguo, SHEN Peihong, et al. Research on methods of K-means clustering and recognition for driving style [J]. Automobile Technology, 2018(12): 8-12.

LIN Xinyou, LI Kuiliang, WANG Liming. A driving-style-oriented adaptive control strategy based PSO-fuzzy expert algorithm for a plug-in hybrid electric vehicle [J]. Expert Systems with Applications, 2022, 201: 117236.

施德华, 容香伟, 汪少华, 等. 基于功率比的混合动力汽车模糊自适应等效燃油消耗最小策略研究 [J]. 西安交通大学学报, 2022, 56(1): 12-21.

SHI Dehua, RONG Xiangwei, WANG Shaohua, et al. Fuzzy adaptive equivalent consumption minimization strategy for hybrid electric vehicle based on power ratio [J]. Journal of Xi'an Jiaotong University, 2022, 56(1): 12-21.

TIAN Xiang, CAI Yingfeng, SUN Xiaodong, et al. An adaptive ECMS with driving style recognition for energy optimization of parallel hybrid electric buses [J]. Energy, 2019, 189: 116151.

ONORI S, SERRAO L, RIZZONI G. Adaptive equivalent consumption minimization strategy for hybrid electric vehicles [C]//ASME 2010 Dynamic Systems and Control Conference. New York, USA: ASME, 2010: 499-505.

WANG Zexing, HE Hongwen, PENG Jiankun, et al. A comparative study of deep reinforcement learning based energy management strategy for hybrid electric vehicle [J]. Energy Conversion and Management, 2023, 293: 117442.

KONG Yan, XU Nan, LIU Qiao, et al. A data-driven energy management method for parallel PHEVs based on action dependent heuristic dynamic programming(ADHDP)model [J]. Energy, 2023, 265: 126306.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Fuzzy Adaptive Equivalent Consumption Minimization Strategy for Hybrid Electric Vehicle Based on Power Ratio

Control Strategy for Battery Thermal Management System Using Deep Reinforcement Learning

Hierarchical Twin-Delayed Policy Gradient Reinforcement Learning for Intelligent Cooperative Control of Aircraft

Finite-Time Prescribed Performance Coordinated Control for Power-Split Hybrid Electric Vehicle Systems

Battery Thermal Management System Control Strategy Based on Deep Reinforcement Learning

Related Author

陈龙

汪少华

容香伟

施德华

XIE Yi

HUANG Kunjie

ZOU Zihao

HUANG Chi

Related Institution

Ministry of Education Key Laboratory of Advanced Manufacture Technology for Automobile Parts, Chongqing University of Technology

School of Mechanical and Transportation Engineering, Chongqing University

School of Energy and Power Engineering, Chongqing University

Key Laboratory of Low-Grade Energy Utilization Technologies and Systems of Ministry of Education, Chongqing University

School of Mechanical Engineering, Tianjin University

⁰