Deep Reinforcement Learning Control Strategy for Primary Frequency Regulation ofEnergy Storage Assisted Thermal Power Units

WANG Jianbo; SUN Ran; LIU Zhongkai; ZHANG Xiaoke; GUO Hongzuo; HU Huaizhong

doi:10.7652/xjtuxb202406017

您当前的位置：

首页 >

文章列表页 >

Deep Reinforcement Learning Control Strategy for Primary Frequency Regulation ofEnergy Storage Assisted Thermal Power Units

更新时间：2025-07-28

- Deep Reinforcement Learning Control Strategy for Primary Frequency Regulation ofEnergy Storage Assisted Thermal Power Units
- Vol. 58, Issue 6, Pages: 186-192(2024)
- 作者机构：
  
  1. 国网河南省电力公司,郑州,450000
  2. 西安交通大学自动化科学与工程学院,西安,710100
  3. 国网河南省电力公司电力科学研究院,郑州,450052
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202406017
  CLC： TM734
- Online First：10 June 2024，
  
  Published：2024
- 稿件说明：
移动端阅览
WANG Jianbo, SUN Ran, LIU Zhongkai, et al. Deep Reinforcement Learning Control Strategy for Primary Frequency Regulation ofEnergy Storage Assisted Thermal Power Units[J]. 2024, 58(6): 186-192.
DOI：

WANG Jianbo, SUN Ran, LIU Zhongkai, et al. Deep Reinforcement Learning Control Strategy for Primary Frequency Regulation ofEnergy Storage Assisted Thermal Power Units[J]. 2024, 58(6): 186-192. DOI： 10.7652/xjtuxb202406017.

摘要

为充分挖掘火储联合系统中储能单元的一次调频潜力

同时减少火电机组调频频度

提出了一种基于双延迟深度确定性策略梯度算法的储能辅助火电机组一次调频控制策略。搭建了典型含有火储联合调频系统的区域电网一次调频模型; 以提高电网频率控制效果、维持储能荷电状态(SOC)平稳、减少火电机组调频动作次数等多项目标建立优化问题

综合考虑电网运行状态及调频动作的约束

将储能辅助火电机组一次调频问题建模为一个马尔可夫决策过程; 使用双延迟深度确定性策略梯度算法进行优化问题求解。典型调频任务场景的仿真结果表明

与传统火储联合调频控制算法相比

所提出的储能辅助火电机组智能调频控制策略可使电网频率偏差均值降低约10.4%、火电机组的调频动作次数减少约56.2%

验证了该策略能在有效释放火储联合系统的调频潜力

同时大幅减少火电机组一次调频频次。

Abstract

In order to fully tap the potential of primary frequency modulation in the energy storage unit of the thermal-storage combined system and reduce the frequency regulation of thermal power units

a twin delayed deep deterministic policy gradient algorithm based energy storage assisted primary frequency regulation control strategy for thermal power unit is proposed. A typical regional power grid primary frequency regulation model containing thermal-storage combined frequency regulation system is established. Optimization problems with multiple objectives such as improving the frequency control effect of the power grid

maintaining a stable state of charge(SOC)

and reducing the frequency regulation actions of thermal power unit are set. Taking into account the constraints of power grid operation status and frequency regulation actions

the primary frequency regulation problem of thermal power unit assisted by energy storage is modeled as a Markov decision process. Twin delayed deep deterministic policy gradient algorithm is used for optimization problem solving. The simulation results of typical frequency regulation task scenarios show that compared with traditional frequency regulation system control algorithms

the proposed energy storage assisted intelligent frequency regulation control strategy for thermal power units can reduce the average frequency difference of the power grid by about 10.4%

and reduce the frequency regulation actions of thermal power unit by about 56.2%. This verifies that the strategy can effectively release the frequency regulation potential of the thermal-storage combined system while significantly reducing frequency regulation of thermal power units.

关键词

Keywords

references

张剑云, 李明节. 新能源高渗透的电力系统频率特性分析 [J]. 中国电机工程学报, 2020, 40(11): 3498-3506.

ZHANG Jianyun, LI Mingjie. Analysis of the frequency characteristic of the power systems highly penetrated by new energy generation [J]. Proceedings of the CSEE, 2020, 40(11): 3498-3506.

叶林, 王凯丰, 赖业宁, 等. 低惯量下电力系统频率特性分析及电池储能调频控制策略综述 [J]. 电网技术, 2023, 47(2): 446-462.

YE Lin, WANG Kaifeng, LAI Yening, et al. Review of frequency characteristics analysis and battery energy storage frequency regulation control strategies in power system under low inertia level [J]. Power System Technology, 2023, 47(2): 446-462.

LIU Ju, YAO Wei, WEN Jinyu, et al. Impact of power grid strength and PLL parameters on stability of grid-connected DFIG wind farm [J]. IEEE Transactions on Sustainable Energy, 2020, 11(1): 545-557.

张小科, 王子杰, 夏大伟, 等. 一种面向深度调峰运行火电机组的一次调频能力建模新方法 [J]. 电网技术, 2022, 46(12): 4947-4953.

ZHANG Xiaoke, WANG ZI Jie, XIA Dawei, et al. New modeling for primary frequency regulation capability of thermal power units under deep peak regulation [J]. Power System Technology, 2022, 46(12): 4947-4953.

赵东元, 胡楠, 傅靖, 等. 提升新能源电力系统灵活性的中国实践及发展路径研究 [J]. 电力系统保护与控制, 2020, 48(24): 1-8.

ZHAO Dongyuan, HU Nan, FU Jing, et al. Research on the practice and road map of enhancing the flexibility of a new generation power system in China [J]. Power System Protection and Control, 2020, 48(24): 1-8.

卓振宇, 张宁, 谢小荣, 等. 高比例可再生能源电力系统关键技术及发展挑战 [J]. 电力系统自动化, 2021, 45(9): 171-191.

ZHUO Zhenyu, ZHANG Ning, XIE Xiaorong, et al. Key technologies and developing challenges of power system with high proportion of renewable energy [J]. Automation of Electric Power Systems, 2021, 45(9): 171-191.

张程铭, 柳璐, 程浩忠, 等. 考虑频率安全的电力系统规划与运行优化研究综述与展望 [J]. 电网技术, 2022, 46(1): 250-264.

ZHANG Chengming, LIU Lu, CHENG Haozhong, et al. Review and prospects of planning and operation optimization for electrical power systems considering frequency security [J]. Power System Technology, 2022, 46(1): 250-264.

刘畅, 卓建坤, 赵东明, 等. 利用储能系统实现可再生能源微电网灵活安全运行的研究综述 [J]. 中国电机工程学报, 2020, 40(1): 1-18.

LIU Chang, ZHUO Jiankun, ZHAO Dongming, et al. A review on the utilization of energy storage system for the flexible and safe operation of renewable energy microgrids [J]. Proceedings of the CSEE, 2020, 40(1): 1-18.

李欣然, 邓涛, 黄际元, 等. 储能电池参与电网快速调频的自适应控制策略 [J]. 高电压技术, 2017, 43(7): 2362-2369.

LI Xinran, DENG Tao, HUANG Jiyuan, et al. Battery energy storage systems self-adaptation control strategy in fast frequency regulation [J]. High Voltage Engineering, 2017, 43(7): 2362-2369.

邓霞, 孙威, 肖海伟. 储能电池参与一次调频的综合控制方法 [J]. 高电压技术, 2018, 44(4): 1157-1165.

DENG Xia, SUN Wei, XIAO Haiwei. Integrated control strategy of battery energy storage system in primary frequency regulation [J]. High Voltage Engineering, 2018, 44(4): 1157-1165.

李欣然, 崔曦文, 黄际元, 等. 电池储能电源参与电网一次调频的自适应控制策略 [J]. 电工技术学报, 2019, 34(18): 3897-3908.

LI Xinran, CUI Xiwen, HUANG Jiyuan, et al. The Self-Adaption control strategy of energy storage batteries participating in the primary frequency regulation [J]. Transactions of China Electrotechnical Society, 2019, 34(18): 3897-3908.

吴启帆, 宋新立, 张静冉, 等. 电池储能参与电网一次调频的自适应综合控制策略研究 [J]. 电网技术, 2020, 44(10): 3829-3836.

WU Qifan, SONG Xinli, ZHANG Jingran, et al. Study on self-adaptation comprehensive strategy of battery energy storage in primary frequency regulation of power grid [J]. Power System Technology, 2020, 44(10): 3829-3836.

李军徽, 侯涛, 穆钢, 等. 基于权重因子和荷电状态恢复的储能系统参与一次调频策略 [J]. 电力系统自动化, 2020, 44(19): 63-72.

LI Junhui, HOU Tao, MU Gang, et al. Primary frequency regulation strategy with energy storage system based on weight factors and state of charge recovery [J]. Automation of Electric Power Systems, 2020, 44(19): 63-72.

王育飞, 杨铭诚, 薛花, 等. 计及SOC的电池储能系统一次调频自适应综合控制策略 [J]. 电力自动化设备, 2021, 41(10): 192-198, 219.

WANG Yufei, YANG Mingcheng, XUE Hua, et al. Self-adaptive integrated control strategy of battery energy storage system considering SOC for primary frequency regulation [J]. Electric Power Automation Equipment, 2021, 41(10): 192-198, 219.

李军徽, 高卓, 应鸿, 等. 基于动态下垂系数与SOC基点的储能一次调频控制策略 [J]. 电力系统保护与控制, 2021, 49(5): 1-10.

LI Junhui, GAO Zhuo, YING Hong, et al. Primary frequency regulation control strategy of energy storage based on dynamic droop coefficient and SOC reference [J]. Power System Protection and Control, 2021, 49(5): 1-10.

马智慧, 李欣然, 谭庄熙, 等. 考虑储能调频死区的一次调频控制方法 [J]. 电工技术学报, 2019, 34(10): 2102-2115.

MA Zhihui, LI Xinran, TAN Zhuangxi, et al. Integrated control of primary frequency regulation considering dead band of energy storage [J]. Transactions of China Electrotechnical Society, 2019, 34(10): 2102-2115.

张峰, 游欢欢, 丁磊. 新能源一次调频死区影响机理建模及系数修正策略 [J]. 电力系统自动化, 2023, 47(6): 158-167.ZHANG Feng, YOU Huanhuan, DING Lei. Influential mechanism modelling of dead band in primary frequency regulation of renewable energy and its coefficient correction strategy [J]. Automation of Electric Power Systems, 2023,47(6): 158-167.

HU Junyan, NIU Hanlin, CARRASCO J, et al. Voronoi-based multi-robot autonomous exploration in unknown environments via deep reinforcement learning [J]. IEEE Transactions on Vehicular Technology, 2020, 69(12): 14413-14423.

FUJIMOTO S,VAN HOOF H, MEGER D. Addressing function approximation error in actor-critic methods [C]//Proceedings of the 35th International Conference on Machine Learning. Chia Laguna Resort, Sardinia, Italy: PMLR, 2018: 1587-1596.

LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[EB/OL].(2019-07-05)[2023-06-06]. https://arxiv.org/abs/1509.02971.

ZHANG Yingjun, ZHAO Changhong, TANG Wanrong, et al. Profit-maximizing planning and control of battery energy storage systems for primary frequency control [J]. IEEE Transactions on Smart Grid, 2018, 9(2): 712-723.

刘英培, 田仕杰, 梁海平, 等. 考虑SOC的电池储能系统一次调频策略研究 [J]. 电力系统保护与控制, 2022, 50(13): 107-118.

LIU Yingpei, TIAN Shijie, LIANG Haiping, et al. Control strategy of a battery energy storage system considering SOC in primary frequency regulation of power grid [J]. Power System Protection and Control, 2022, 50(13): 107-118.

修连成, 熊连松, 康志亮, 等. 运用下垂控制的并网储能系统惯量阻尼特性分析 [J]. 西安交通大学学报, 2018, 52(12): 112-120.

XIU Liancheng, XIONG Liansong, KANG Zhiliang, et al. Analysis on dynamic characteristics of grid-tied energy storage system with droop control [J]. Journal of Xi'an Jiaotong University, 2018, 52(12): 112-120.

FRADLEY J, PREECE R, BARNES M.Fast frequency response from MMC-HVDC with varying dead band implementation [J]. IFAC-PapersOnLine, 2018, 51(28): 49-54.

LIU Muyang, BIZZARRI F, BRAMBILLA A M, et al. On the impact of the dead-band of power system stabilizers and frequency regulation on power system stability [J]. IEEE Transactions on Power Systems, 2019, 34(5): 3977-3979.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Intelligent Deformation Decision-Making Method for Morphing Glide Vehicle

A Parallel Service Function Chain Deployment Method Combining Federated Learning and Deep Reinforcement Learning

A Cloud-Edge Collaborative State of Charge Estimation Method for Lithium-Ion Batteries in Mining Environments Integrating Deep Learning and Ampere-Hour Integration

Intelligent Deformation Decision-making Method for Morphing Glide Vehicle with Deep Reinforcement Learning

Bidirectional Balancing and Model Predictive Control Optimization for Lithium Batteries

Related Author

WANG Taojun

MENG Fanyi

CHEN Gang

LIU Guangyuan

DU Jie

PANG Ziyuan

ZHANG Xiangfeng

SHEN Yong

Related Institution

School of Aerospace Engineering, Xi'an Jiaotong University

Shijiazhuang Key Laboratory of Artificial Intelligence

Hebei Provincial Key Laboratory of Electromagnetic Environmental Effects and Information Processing

School of Information Science and Technology, Shijiazhuang Tiedao University

School of Mechanical and Automotive Engineering, Xiamen University of Technology

⁰