采用频域约束强化学习的微电网异构混合储能调度方法

崔菲菲; 安豆; 杨清宇; 任志刚

doi:10.7652/xjtuxb202611006

您当前的位置：

首页 >

文章列表页 >

采用频域约束强化学习的微电网异构混合储能调度方法

更新时间：2026-11-09

- 采用频域约束强化学习的微电网异构混合储能调度方法
- Reinforcement Learning-Based Dispatching Method with Frequency-Domain Constraints for Heterogeneous Hybrid Energy Storage Systems in Microgrids
- 西安交通大学学报 2026年60卷第11期页码：59-70
- 作者机构：
  
  西安交通大学自动化科学与工程学院，710049，西安
  西安交通大学精密微纳制造技术全国重点实验室，710049，西安
- 作者简介：
  
  崔菲菲（1999—），女，博士生；
  安豆（通信作者），男，教授，博士生导师。
- 基金信息：
  
  国家重点研发计划资助项目(2024YFB4206500);国家自然科学基金资助项目(62573344;62373297)
- DOI：10.7652/xjtuxb202611006
  中图分类号： TP273
- 收稿：2026-03-21，
  
  网络首发：2026-08-12，
  
  纸质出版：2026-11-10
- 稿件说明：
移动端阅览
崔菲菲, 安豆, 杨清宇, 等. 采用频域约束强化学习的微电网异构混合储能调度方法[J/OL]. 西安交通大学学报,2026,60 (11):59-70. https://doi.org/10.7652/xjtuxb202611006.

CUI Feifei, AN Dou, YANG Qingyu, et al. Reinforcement Learning-Based Dispatching Method with Frequency-Domain Constraints for Heterogeneous Hybrid Energy Storage Systems in Microgrids[J/OL]. Journal of Xi'an Jiaotong University,2026,60 (11):59-70. https://doi.org/10.7652/xjtuxb202611006.
崔菲菲, 安豆, 杨清宇, 等. 采用频域约束强化学习的微电网异构混合储能调度方法[J/OL]. 西安交通大学学报,2026,60 (11):59-70. https://doi.org/10.7652/xjtuxb202611006. DOI：

CUI Feifei, AN Dou, YANG Qingyu, et al. Reinforcement Learning-Based Dispatching Method with Frequency-Domain Constraints for Heterogeneous Hybrid Energy Storage Systems in Microgrids[J/OL]. Journal of Xi'an Jiaotong University,2026,60 (11):59-70. https://doi.org/10.7652/xjtuxb202611006. DOI：

摘要

在微电网源、荷两侧波动加剧的背景下，针对传统强化学习算法隐式引导异构储能调度易导致策略同质化、协同分工失效的问题，提出一种基于频域约束的深度强化学习协同优化调度方法。首先，面向微电网压缩空气储能与电池储能异构混合储能系统，建立马尔可夫决策过程模型以刻画调度系统的状态演变特征与序贯决策特性；其次，基于双延迟深度确定性策略梯度强化学习算法，将谱正则化频域约束惩罚项引入算法策略网络梯度中，以引导电池储能吸收高波动功率，压缩空气储能跟踪低频基波，实现二者差异化分工。仿真结果表明，与主流深度强化学习基准算法相比，所提算法在典型周调度内，可将综合运行成本降低10.0%，并网功率跟踪误差降低约68.5%，并网功率越线次数减少约92.1%，压缩空气储能机械寿命指标与电池储能等效全循环次数分别下降92%和53.8%。该算法可兼顾微电网系统经济性、电网安全稳定性以及底层储能设备的寿命友好性。

Abstract

In the context of intensified fluctuations on both the generation and load sides of microgrids

conventional deep reinforcement learning algorithms often lead to strategy homogenization and failure of synergistic division of labor due to their implicit guidance of heterogeneous energy storage dispatch.To address these drawbacks

a coordinated optimal dispatch method based on frequencydomain-constrained deep reinforcement learning is proposed.First

tailored to the heterogeneous hybrid energy storage system comprising compressed air energy storage and battery energy storage systems in microgrids

a Markov decision process model is established to characterize the state evolution features and sequential decision-making properties of the dispatch system.Next

based on the twin delayed deep deterministic policy gradient (TD3 ) reinforcement learning algorithm

a spectral-regularized frequency-domain constraint penalty term is incorporated into the gradient of the algorithm's policy network to guide the battery energy storage system to absorb high-fluctuation power and the compressed air energy storage system to track the lowfrequency fundamental wave

thereby achieving a differentiated division of labor between the two.Simulation results demonstrate that

compared with mainstream deep reinforcement learning benchmark algorithms

the proposed algorithm can reduce the comprehensive operating cost by 10.0%

decrease the grid-connected power tracking error by approximately 68.5%

and lower the number of grid power limit violations by 92.1% in a typical weekly dispatch scenario. Moreover

the mechanical life index of the compressed air energy storage and the equivalent full cycle number of the battery energy storage are decreased by 92% and 53.8%

respectively.This algorithm can balance the economic efficiency of the microgrid system

the safety and stability of the power grid

and the lifespan friendliness of the underlying energy storage devices.

关键词

Keywords

references

Impram S,Varbak Nese S, Oral B.Challenges of renewable energy penetration on power system flexibility:a survey [J]. Energy Strategy Reviews, 2020, 31 :100539.

Zang Tianlei,Wang Shijun,Wang Zian,et al.Integrated planning and operation dispatching of source-grid-loadstorage in a new power system:a coupled socio-cyberphysical perspective [J].Energies,2024,17(12):3013.

Ahmethodžic′ L,Music′ M,Huseinbegovic′ S.Microgrid energy management:classification,review and challenges [J].CSEE Journal of Power and Energy Systems,2023 ,9(4):1425-1438.

Rezaeimozafar M,Monaghan R F D,Barrett E,et al. A review of behind-the-meter energy storage systems in smart grids [J].Renewable and Sustainable Energy Reviews,2022,164:112573.

Rana M M,Uddin M,Sarkar M R,et al.Applications of energy storage systems in power grids with and without renewable energy integration:a comprehensive review [J].Journal of Energy Storage,2023, 68:107811.

Wang Yujie,Wang Li,Li Mince,et al.A review of key issues for control and management in battery and ultra-capacitor hybrid energy storage systems [J]. eTransportation,2020,4:100064.

Fang Ziyu,Shek J K H,Sun Wei.A review of gridconnected hybrid energy storage systems:sizing configurations,control strategies,and future directions [J]. Journal of Energy Storage,2025,118:116226.

Agyekum E B,Abdullah M,Rashid F L,et al.Compressed air energy storage (CAES)systems:technological progress,challenges,and future prospects in renewable energy grids [J].Energy Conversion and Management:Ⅹ,2025 ,28:101402.

Zeynali S,Rostami N,Ahmadian A,et al.Robust multi-objective thermal and electrical energy hub management integrating hybrid battery-compressed air energy storage systems and plug-in-electric-vehicle-based demand response [J].Journal of Energy Storage, 2021 ,35:102265.

Cui Feifei,An Dou,Xi Huan.Integrated energy hub dispatch with a multi-modeCAES-BESShybrid system:an option-based hierarchical reinforcement learning approach [J].Applied Energy,2024,374:123950.

Xiao Gang,Xu Fen,Tong Lianghuai,et al.A hybrid energy storage system based on self-adaptive variational mode decomposition to smooth photovoltaic power fluctuation [J].Journal of Energy Storage,2022,55:105509.

Chu Zhongda,Zhang Ning,Teng Fei.Frequency-constrained resilient scheduling of microgrid:a distributionally robust approach [J].IEEE Transactions on Smart Grid,2021,12(6):4914-4925.

Nakiganda A M,Dehghan S,Markovic U,et al.A stochastic-robust approach for resilient microgrid investment planning under static and transient islanding security constraints [J].IEEE Transactions on Smart Grid,2022,13(3):1774-1788.

Babu V V,Preetha Roselyn J, Sundaravadivel P. Multi-objective genetic algorithm based energy management system considering optimal utilization of grid and degradation of battery storage in microgrid [J]. Energy Reports,2023,9:5992-6005.

陈明昊,孙毅,谢志远.基于双层深度强化学习的园区综合能源系统多时间尺度优化管理[J].电工技术学报,2023,38(7):1864-1881 .

Chen Minghao,Sun Yi,Xie Zhiyuan.The multi-timescale management optimization method for park integrated energy system based on the bi-layer deep reinforcement learning[J].Transactions of China Electrotechnical Society,2023,38(7):1864-1881.

熊珞琳,毛帅,唐漾,等.基于强化学习的综合能源系统管理综述[J].自动化学报,2021 ,47(10):23212340.

Xiong Luolin,Mao Shuai,Tang Yang,et al.Reinforcement learning based integrated energy system management:a survey[J].Acta Automatica Sinica, 2021 ,47(10):2321-2340.

Zheng Yi,Jia Jinhua,An Dou.Energy management for microgrids with hybrid hydrogen-battery storage:a reinforcement learning framework integrated multi-objective dynamic regulation [J].Processes,2025,13(8):2558.

Kumar K,Kwon S,Bae S.Deep reinforcement learning-based control strategy for integration of a hybrid energy storage system in microgrids [J].Journal of Energy Storage,2025,108:114936.

Wang Xin,Zhou Jianshu,Qin Bin,et al.Coordinated control of wind turbine and hybrid energy storage system based on multi-agent deep reinforcement learning for wind power smoothing [J].Journal of Energy Storage,2023,57:106297.

李鹏,钟瀚明,马红伟,等.基于深度强化学习的有源配电网多时间尺度源荷储协同优化调控[J].电工技术学报,2025,40(5):1487-1502.

Li Peng,Zhong Hanming,Ma Hongwei,et al.Multitimescale optimal dispatch of source-load-storage coordination in active distribution network based on deep reinforcement learning[J]. Transactions of China Electrotechnical Society,2025,40(5):1487-1502.

Cui Feifei,An Dou,Teng Shiyang,et al.Cogeneration systems of solar energy integrated with compressed air energy storage systems: a comparative study of various energy recovery strategies [J].Case Studies in Thermal Engineering,2023,51:103521.

Xu Bolun,Zhao Jinye,Zheng Tongxin,et al.Factoring the cycle aging cost of batteries participating in electricity markets [J].IEEE Transactions on Power Systems,2018,33(2):2248-2259.

Chen Junyu,Zhao Yue,Wang Minghao,et al.Multitimescale reward-based DRL energy management for regenerative braking energy storage system [J].IEEE Transactions on Transportation Electrification,2025, 11(3):7488-7500.

Egbomwan O E,Chaoui H,Liu Shichao.A physicsconstrained TD3 algorithm for simultaneous virtual inertia and damping control of grid-connected variable speed DFIG wind turbines [J].IEEE Transactions on Automation Science and Engineering, 2025, 22:958-969.

高绪冲,王朝民,张展泳,等.融合混合储能的微电网多时间尺度优化调度策略[J/OL].南方电网技术, 2026-01-05. https://link.cnki.net/urlid/44.1643.tk. 20260105.1429.016.

Gao Xuchong,Wang Chaomin,Zhang Zhanyong,et al. Multi-time-scale optimal scheduling strategy for Microgrids integrating hybrid energy storage[J/OL]. Southern Power System Technology,2026-01-05.https://link.cnki.net/urlid/44.1643.tk.20260105.1429.016.

Li Li,Wang Jing,Zhong Xiaoyi,et al.Combined multi-objective optimization and agent-based modeling for a 100% renewable island energy system considering power-to-gas technology and extreme weather conditions [J].Applied Energy,2022,308:118376.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向变形滑翔飞行器的智能变形决策方法

联合联邦学习与深度强化学习的并行服务功能链部署算法

面向具身智能体连续控制任务的可解释线性因果策略模型

基于深度强化学习的变形滑翔飞行器智能变形决策方法

三相四桥臂并联逆变器的零序电流建模与控制