考虑驾驶风格的混合动力汽车强化学习能量管理策略

施德华; 袁超; 汪少华; 周卫琪; 陈龙

doi:10.7652/xjtuxb202410005

您当前的位置：

首页 >

文章列表页 >

考虑驾驶风格的混合动力汽车强化学习能量管理策略

专题电动车辆先进电池管理 | 更新时间：2025-07-28

- 考虑驾驶风格的混合动力汽车强化学习能量管理策略
- Reinforcement Learning-Based Energy Management Strategy Considering Driving Style for Hybrid Electric Vehicle
- 西安交通大学学报 2024年58卷第10期页码：51-62
- 作者机构：
  
  1. 江苏大学汽车工程研究院,江苏,镇江,212013
  2. 江苏省新能源汽车电驱动系统与智能控制工程研究中心,江苏,镇江,212013
- 作者简介：
  
  : 2024-01-18。作者简介: 施德华(1989—),男,副教授,硕士生导师。基金项目: 国家自然科学基金资助项目(51905219)
- 基金信息：
  
  : 2024-01-18。作者简介: 施德华(1989—),男,副教授,硕士生导师。基金项目: 国家自然科学基金资助项目(51905219);中国博士后科学基金资助项目(2023M731444);中国科协青年人才托举工程资助项目(2020QNRC001)。
- DOI：10.7652/xjtuxb202410005
  中图分类号： U463.2
- 网络首发：2024-10-10，
  
  纸质出版：2024
- 稿件说明：
移动端阅览
施德华, 袁超, 汪少华, 等. 考虑驾驶风格的混合动力汽车强化学习能量管理策略[J]. 西安交通大学学报, 2024,58(10):51-62.

SHI Dehua, YUAN Chao, WANG Shaohua, et al. Reinforcement Learning-Based Energy Management Strategy Considering Driving Style for Hybrid Electric Vehicle[J]. 2024, 58(10): 51-62.
施德华, 袁超, 汪少华, 等. 考虑驾驶风格的混合动力汽车强化学习能量管理策略[J]. 西安交通大学学报, 2024,58(10):51-62. DOI： 10.7652/xjtuxb202410005.

SHI Dehua, YUAN Chao, WANG Shaohua, et al. Reinforcement Learning-Based Energy Management Strategy Considering Driving Style for Hybrid Electric Vehicle[J]. 2024, 58(10): 51-62. DOI： 10.7652/xjtuxb202410005.

摘要

为了提升混合动力汽车能量管理策略对不同风格驾驶员的适应性

基于深度强化学习和等效燃油消耗最小策略(equivalent consumption minimization strategy

ECMS)

提出一种考虑驾驶风格的混合动力汽车能量管理策略。通过实车试验采集驾驶员行驶数据

基于采集数据进行驾驶员驾驶风格的聚类分析

建立驾驶风格识别模型; 构建基于强化学习和ECMS的能量管理策略

将驾驶风格系数作为强化学习状态变量

利用多种驾驶风格的组合工况训练深度确定性策略梯度智能体

获取不同工况和驾驶风格下ECMS等效因子

采用ECMS求解最优发动机、电机转矩分配以及变速箱挡位; 搭建硬件在环测试平台

并基于实际采集的不同驾驶员驾驶数据构建测试工况

验证所提出控制策略的有效性。研究结果表明

相较于基于规则策略、基于等效因子比例修正的自适应ECMS以及DRL-SAC策略

提出的考虑驾驶风格的强化学习能量管理策略使整车能量消耗分别降低16.35%、11.11%和7.56%

所提控制策略的有效性得到了验证。

Abstract

A considering driving style energy management strategy is proposed to enhance the adaptability of energy management strategies for different driving styles in hybrid electric vehicles. The strategy combines deep reinforcement learning with the equivalent consumption minimization strategy(ECMS). Real vehicle experiments are conducted to collect driving data

which is then subjected to clustering analysis to identify distinct driving styles. A driving style recognition model is developed based on this data. The energy management strategy is built using reinforcement learning and ECMS

with driving style coefficients serving as the reinforcement learning state variables. A deep deterministic policy gradient agent is trained using various combinations of driving styles and operating conditions to determine ECMS equivalent factors for different driving styles and conditions. The ECMS is employed to optimize the engine

motor torque allocation

and gearbox gear selection. To validate the effectiveness of the proposed control strategy

a hardware in the loop testing platform is constructed

and test scenarios are generated using real driving data from different drivers. The research findings demonstrate that the reinforcement learning-based energy management strategy considering driving style reduces overall vehicle energy consumption by 16.35%

11.11%

and 7.56% compared with rule-based strategy

equivalent factor ratio correction-based adaptive ECMS

and DRL-SAC strategy

respectively. The effectiveness of the proposed control strategy is successfully validated.

关键词

Keywords

references

王跃飞, 王志, 孙睿, 等. 基于驾驶意图多步预测的智能网联HEV等效排放最小控制策略 [J]. 机械工程学报, 2023, 59(18): 271-282.

WANG Yuefei, WANG Zhi, SUN Rui, et al. Equivalent emission minimization strategy of intelligent connected HEV based on multi-step prediction of driving intention [J]. Journal of Mechanical Engineering, 2023, 59(18): 271-282.

黄康, 王强, 邱明明, 等. 考虑模式切换频率的多模式混合动力汽车参数优化 [J]. 西安交通大学学报, 2019, 53(7): 99-107.

HUANG Kang, WANG Qiang, QIU Mingming, et al. Parameter optimization of multi-mode hybrid vehicle considering mode-switching frequency [J]. Journal of Xi'an Jiaotong University, 2019, 53(7): 99-107.

YANG Chao, DU Xuelong, WANG Weida, et al. Variable optimization domain-based cooperative energy management strategy for connected plug-in hybrid electric vehicles [J]. Energy, 2024, 290: 130206.

黄硕, 李亮, 杨超, 等. 基于规则修正的同轴并联混合动力客车瞬时优化能量分配策略 [J]. 机械工程学报, 2014, 50(20): 113-121.

HUANG Shuo, LI Liang, YANG Chao, et al. Rule correction-based instantaneous optimal energy management strategy for single-shaft parallel hybrid electric bus [J]. Journal of Mechanical Engineering, 2014, 50(20): 113-121.

DU Guodong, ZOU Yuan, ZHANG Xudong, et al. Deep reinforcement learning based energy management for a hybrid electric vehicle [J]. Energy, 2020, 201: 117591.

王文彬, 田韶鹏, 郑青星, 等. 基于FA的等效燃油消耗最小控制策略优化 [J]. 江苏大学学报(自然科学版), 2022, 43(2): 147-153.

WANG Wenbin, TIAN Shaopeng, ZHENG Qingxing, et al. Optimization of equivalent fuel consumption minimization strategy based on firefly algorithm [J]. Journal of Jiangsu University(Natural Science Edition), 2022, 43(2): 147-153.

CHEN Yao, WANG Ke, LU J J. Feature selection for driving style and skill clustering using naturalistic driving data and driving behavior questionnaire [J]. Accident Analysis Prevention, 2023, 185: 107022.

ZHANG Zhen, ZHANG Tiezhu, HONG Jichao, et al. Energy management strategy of a novel electric-hydraulic hybrid vehicle based on driving style recognition [J]. Sustainable Energy Fuels, 2023, 7(2): 420-430.

TIAN Xiang, CAI Yingfeng, SUN Xiaodong, et al. Incorporating driving style recognition into MPC for energy management of plug-in hybrid electric buses [J]. IEEE Transactions on Transportation Electrification, 2023, 9(1): 169-181.

GUO Qiuyi, ZHAO Zhiguo, SHEN Peihong, et al. Adaptive optimal control based on driving style recognition for plug-in hybrid electric vehicle [J]. Energy, 2019, 186: 115824.

ZHU Zhaoxuan, LIU Yuxing, CANOVA M. Energy management of hybrid electric vehicles via deep Q-networks [C]//2020 American Control Conference(ACC). Piscataway, NJ, USA: IEEE, 2020: 3077-3082.

LEE W, JEOUNG H, PARK D, et al. A real-time intelligent energy management strategy for hybrid electric vehicles using reinforcement learning [J]. IEEE Access, 2021, 9: 72759-72768.

XU Bin, HOU Jun, SHI Junzhe, et al. Learning time reduction using warm-start methods for a reinforcement learning-based supervisory control in hybrid electric vehicle applications [J]. IEEE Transactions on Transportation Electrification, 2021, 7(2): 626-635.

HU Dong, ZHANG Yuanyuan. Deep reinforcement learning based on driver experience embedding for energy management strategies in hybrid electric vehicles [J]. Energy Technology, 2022, 10(6): 2200123.

WANG Hanchen, YE Yiming, ZHANG Jiangfeng, et al. A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle [J]. Energy, 2023, 266: 126497.

HE Kun, QIN Dongchen, CHEN Jiangyi, et al. Adaptive equivalent consumption minimization strategy for fuel cell buses based on driving style recognition [J]. Sustainability, 2023, 15(10): 7781.

YANG Sen, WANG Wenshuo, ZHANG Fengqi, et al. Driving-style-oriented adaptive equivalent consumption minimization strategies for HEVs [J]. IEEE Transactions on Vehicular Technology, 2018, 67(10): 9249-9261.

GONG Changchao, HU Minghui, LI Shuxian, et al. Equivalent consumption minimization strategy of hybrid electric vehicle considering the impact of driving style [J]. Proceedings of the Institution of Mechanical Engineers: Part D Journal of Automobile Engineering, 2019, 233(10): 2610-2623.

王旭, 马菲, 廖小棱, 等. 基于多分类监督学习的驾驶风格特征指标筛选 [J]. 交通信息与安全, 2022, 40(1): 162-168.

WANG Xu, MA Fei, LIAO Xiaoleng, et al. Feature selection for recognition of driving styles based on multi-classification and supervised learning [J]. Journal of Transport Information and Safety, 2022, 40(1): 162-168.

李经纬, 赵治国, 沈沛鸿, 等. 驾驶风格K-means聚类与识别方法研究 [J]. 汽车技术, 2018(12): 8-12.

LI Jingwei, ZHAO Zhiguo, SHEN Peihong, et al. Research on methods of K-means clustering and recognition for driving style [J]. Automobile Technology, 2018(12): 8-12.

LIN Xinyou, LI Kuiliang, WANG Liming. A driving-style-oriented adaptive control strategy based PSO-fuzzy expert algorithm for a plug-in hybrid electric vehicle [J]. Expert Systems with Applications, 2022, 201: 117236.

施德华, 容香伟, 汪少华, 等. 基于功率比的混合动力汽车模糊自适应等效燃油消耗最小策略研究 [J]. 西安交通大学学报, 2022, 56(1): 12-21.

SHI Dehua, RONG Xiangwei, WANG Shaohua, et al. Fuzzy adaptive equivalent consumption minimization strategy for hybrid electric vehicle based on power ratio [J]. Journal of Xi'an Jiaotong University, 2022, 56(1): 12-21.

TIAN Xiang, CAI Yingfeng, SUN Xiaodong, et al. An adaptive ECMS with driving style recognition for energy optimization of parallel hybrid electric buses [J]. Energy, 2019, 189: 116151.

ONORI S, SERRAO L, RIZZONI G. Adaptive equivalent consumption minimization strategy for hybrid electric vehicles [C]//ASME 2010 Dynamic Systems and Control Conference. New York, USA: ASME, 2010: 499-505.

WANG Zexing, HE Hongwen, PENG Jiankun, et al. A comparative study of deep reinforcement learning based energy management strategy for hybrid electric vehicle [J]. Energy Conversion and Management, 2023, 293: 117442.

KONG Yan, XU Nan, LIU Qiao, et al. A data-driven energy management method for parallel PHEVs based on action dependent heuristic dynamic programming(ADHDP)model [J]. Energy, 2023, 265: 126306.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于功率比的混合动力汽车模糊自适应等效燃油消耗最小策略研究

应用深度强化学习的电池热管理系统控制策略

面向飞行器智能协同控制的分层双时延策略梯度强化学习方法

功率分流式混合动力系统有限时间预设性能协调控制

应用深度强化学习的电池热管理系统控制策略