采用多智能体强化学习的多传感器协同控制策略

张哲源; 冯文晖; 连峰; 刘霞; 阴志鹏

doi:10.7652/xjtuxb202611007

您当前的位置：

首页 >

文章列表页 >

采用多智能体强化学习的多传感器协同控制策略

更新时间：2026-11-09

- 采用多智能体强化学习的多传感器协同控制策略
- Multi-Sensor Cooperative Control Strategy Based on Multi-Agent Reinforcement Learning
- 西安交通大学学报 2026年60卷第11期页码：71-83
- 作者机构：
  
  西安交通大学电信学部，710049，西安
  河北新天科创新能源技术有限公司，075000，河北张家口
- 作者简介：
  
  张哲源（2001—），男，硕士生；
  连峰（通信作者），男，教授，博士生导师。
- 基金信息：
  
  国家自然科学基金资助项目(62173266)
- DOI：10.7652/xjtuxb202611007
  中图分类号： TP212
- 收稿：2026-03-30，
  
  网络首发：2026-08-12，
  
  纸质出版：2026-11-10
- 稿件说明：
移动端阅览
张哲源, 冯文晖, 连峰, 等. 采用多智能体强化学习的多传感器协同控制策略[J/OL]. 西安交通大学学报,2026,60 (11):71-83. https://doi.org/10.7652/xjtuxb202611007.

ZHANG Zheyuan, FENG Wenhui, LIAN Feng, et al. Multi-Sensor Cooperative Control Strategy Based on Multi-Agent Reinforcement Learning[J/OL]. Journal of Xi'an Jiaotong University,2026,60 (11):71-83. https://doi.org/10.7652/xjtuxb202611007.
张哲源, 冯文晖, 连峰, 等. 采用多智能体强化学习的多传感器协同控制策略[J/OL]. 西安交通大学学报,2026,60 (11):71-83. https://doi.org/10.7652/xjtuxb202611007. DOI：

ZHANG Zheyuan, FENG Wenhui, LIAN Feng, et al. Multi-Sensor Cooperative Control Strategy Based on Multi-Agent Reinforcement Learning[J/OL]. Journal of Xi'an Jiaotong University,2026,60 (11):71-83. https://doi.org/10.7652/xjtuxb202611007. DOI：

摘要

针对复杂对抗环境下多传感器系统在任务执行与威胁规避过程中的协同决策问题，提出一种采用多智能体强化学习的异构传感器协同控制策略。首先，构建包含主传感器、次传感器、威胁目标和固定监视目标的二维对抗场景，建立传感器机动、辐射控制、目标探测以及威胁目标锁定与命中机制模型，并将多传感器协同管理问题建模为多智能体部分可观测马尔可夫决策过程。其次，在集中训练、分散执行框架下，采用多智能体近端策略优化（MAPPO）算法实现协同决策，其中主传感器通过联合优化平台机动策略与辐射参数（功率与波束指向）完成穿越与监视任务，次传感器通过机动协同与辐射诱引分担主传感器威胁风险。最后，针对强对抗和稀疏反馈条件，设计兼顾任务收益与生存风险的奖励函数，并采用三阶段课程学习策略提升策略训练效率。在包含1个主传感器、2个次传感器和3个威胁目标的仿真场景中进行50回合对比评估，结果显示：在覆盖阈值为0.75时，所提方法的任务成功率达到82%，高于威胁感知启发式策略（66%）、独立近端策略优化策略（68%）和随机策略（0）；同时主传感器存活率达到82%，平均覆盖率达到83.5%。所提方法能够有效协调监视收益与辐射暴露风险，提高复杂对抗环境下多传感器系统的任务完成能力与生存能力。

Abstract

To address the cooperative decision-making problem of multi-sensor systems during mission execution and threat avoidance in complex adversarial environments

this paper proposes a heterogeneous sensor cooperative control strategy based on multi-agent reinforcement learning. First

a two-dimensional adversarial scenario is constructed

which includes primary sensors

secondary sensors

threat targets

and stationary surveillance targets.Models for sensor maneuvering

emission control

target detection

and threat lock-on and engagement mechanisms are established.Subsequently

the multi-sensor cooperative management problem is formulated as a multi-agent partially observable Markov decision process.Under the framework of centralized training with decentralized execution (CTDE)

the multi-agent proximal policy optimization (MAPPO) algorithm is employed for cooperative decision-making.In this framework

the primary sensor accomplishes traversal and surveillance tasks by jointly optimizing its platform maneuver strategy and emission parameters (power and beam pointing)

while the secondary sensors share the threat risk of the primary sensor through maneuver coordination and emission-based decoying. To cope with highly contested conditions and sparse feedback

a reward function is designed to balance mission reward and survival risk

and a three-stage curriculum learning strategy is adopted to improve training efficiency and policy convergence.Comparative evaluations were conducted over 50 test episodes in a simulation scenario containing one primary sensor

two secondary sensors

and three threat targets.Results show that

at a coverage threshold of 0.75

the proposed method achieved a mission success rate of 82%

outperforming the threat-aware heuristic policy (66%)

the independent proximal policy optimization (IPPO) policy (68%)

and the random policy (0) . Meanwhile

the survival rate of the primary sensor reached 82%

and the average coverage rate reached 83.5%.The proposed method effectively balances surveillance utility against emission exposure risk

enhancing both the mission completion capability and the survivability of multisensor systems in complex adversarial environments.

关键词

Keywords

references

Bier S G,Rothman P L,Manske R A.Intelligent sensor management for beyond visual range air-to-air combat [C]//Proceedings of the IEEE 1988 National Aerospace and Electronics Conference.Piscataway,NJ,USA:IEEE,1988:264-269.

Ng G W,Ng K H.Sensor management-what,why and how [J].Information Fusion,2000,1(2):67-75.

Bar-Shalom Y,Li Xiaorong.Multitarget-multisensor tracking principles and techniques [M].Storrs,CT, USA:YBS Publishing,1995 .

Hero A O,Castañón D A,Cochran D,et al.Foundations and applications of sensor management [M]. Boston,MA,USA:Springer US,2008.

Zuo Lei,Hu Juan,Sun Hao,et al.Resource allocation for target tracking in multiple radar architectures over lossy networks [J].Signal Processing,2023,208:108973.

Hoang H G,Vo B T.Sensor management for multitarget tracking via multi-Bernoulli filtering [J].Automatica,2014,50(4):1135-1142.

Jiang Meng,Yi Wei,Kong Lingjiang.Multi-sensor control for multi-target tracking using CauchySchwarz divergence [C]//2016 19th International Conference on Information Fusion (FUSION).Piscataway,NJ,USA:IEEE,2016:2059-2066.

Wang Ping,Ma Liang,Xue Kai.Multitarget tracking in sensor networks via efficient information-theoretic sensor selection [J].International Journal of Advanced Robotic Systems,2017,14(5):1729881417728466.

陈辉,韩崇昭.机动多目标跟踪中的传感器控制策略的研究[J].自动化学报,2016,42(4):512-523.

Chen Hui,Han Chongzhao.Research on sensor control strategy for maneuvering multi-target tracking[J]. Acta Automatica Sinica,2016,42(5):512-523.

Castanon D A.Approximate dynamic programming for sensor management [C]//Proceedings of the 36th IEEE Conference on Decision and Control.Piscataway, NJ,USA:IEEE,1997 :1202-1207.

Krishnamurthy V,Djonin D V.Optimal threshold policies for multivariate POMDPs in radar resource management [J].IEEE Transactions on Signal Processing,2009,57(10):3954-3969.

Schöpe M I,DriessenH,Yarovoy A.A constrained POMDP formulation and algorithmic solution for radar resource management in multi-target tracking [J]. Journal of Advances in Information Fusion, 2021, 16(1):31-47.

王增福,杨广宇,金术玲.考虑综合性能最优的非短视快速天基雷达多目标跟踪资源调度算法[J].雷达学报,2024,13(1):253-269.

Wang Zengfu,Yang Guangyu,Jin Shuling.A non-myopic and fast resource scheduling algorithm for multitarget tracking of space-based radar considering opti mal integrated performance[J].Journal of Radars, 2024,13(1):253-269.

徐公国,单甘霖,段修生.采用马氏决策过程和后验克拉美罗下界的多被动式移动传感器长期调度方法[J].西安交通大学学报,2019,53(6):125-133.

Xu Gongguo,Shan Ganlin,Duan Xiusheng.Non-myopic scheduling method of multiple passive mobile sensors based on Markov decision process and posterior Cramér-Rao lower bound[J].Journal of Xi'an Jiaotong University,2019,53(6):125-133.

Mnih V,Kavukcuoglu K,Silver D,et al.Human-level control through deep reinforcement learning [J].Nature,2015,518(7540):529-533.

Charlish A,Hoffmann F.Cognitive radar management [M].Novel Radar Techniques and Applications Volume 2:Waveform Diversity and Cognitive Radar,and Target Tracking and Data Fusion.Stevenage,Hertfordshire,UK:IET,2017:157-193.

Ravier R,Garagic′D,Peskoe J,et al.Online reinforcement learning for autonomous sensor control [C]//2023 IEEE Aerospace Conference.Piscataway,NJ, USA:IEEE,2023 :1-10.

张虹芸,陈辉,张文旭.扩展目标跟踪中基于深度强化学习的传感器管理方法[J].自动化学报,2024,50 (7):1417-1431.

Zhang Hongyun,Chen Hui,Zhang Wenxu.Sensor management method based on deep reinforcement learning in extended target tracking[J].Acta Automatica Sinica,2024,50(7):1417-1431.

闫实,贺静,王跃东,等.基于强化学习的多机协同传感器管理[J].系统工程与电子技术,2020 ,42(8):1726-1733.

Yan Shi,He Jing,Wang Yuedong,et al.Multi-airborne cooperative sensor management based on reinforcement learning[J].Systems Engineering and Electronics,2020,42(8):1726-1733.

Zhang Kaiqing,Yang Zhuoran,Başar T.Multi-agent reinforcement learning:a selective overview of theories and algorithms [M]. Handbook of Reinforcement Learning and Control.Cham,Switzerland:Springer International Publishing,2021:321-384.

Lowe R,Wu Yi,Tamar A,et al.Multi-agent actorcritic for mixed cooperative-competitive environments [C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.Red Hook, NY,USA:Curran Associates Inc.,2017:6382-6393.

Rashid T,Samvelyan M,De Witt C S,et al.Monotonic value function factorisation for deep multi-agent reinforcement learning [J].The Journal of Machine Learning Research,2020,21(1):178.

Yu Chao,Velu A,Vinitsky E,et al.The surprising effectiveness of PPO in cooperative multi-agent games [C]//Proceedings of the 36th International Conference on Neural Information Processing Systems.Red Hook, NY,USA:Curran Associates Inc.,2022:24611-24624.

Lee J,Niyato D,Guan Yongliang,et al.Learning to schedule joint radar-communication with deep multi-agent reinforcement learning [J].IEEE Transactions on Vehicular Technology,2022,71(1):406-422.

Li Xiaoyang,Wang Teng,Wang Yongkun,et al.Intelligent decision-making algorithm for multi-UAV radar cooperative guided search task based on multiagent reinforcement learning [J].IEEE Internet of Things Journal,2025,12(15):31042-31063.

Beatty E,Dong E.Multi-agent reinforcement learning for UAV sensor management [C]//Open Architecture/Open Business Model Net-Centric Systems and Defense Transformation 2023. Bellingham, WA, USA:SPIE,2023:125440 H.

Yu Yue,Liu Mei.Deep reinforcement learning-based multisensor control for labeled multi-Bernoulli filtering [J].IEEE Transactions on Aerospace and Electronic Systems,2025,61(5):13548-13564.

Cai Bingchen,Li Haoran,Zhang Naimin,et al.A cooperative jamming decision-making method based on multi-agent reinforcement learning [J].Autonomous Intelligent Systems,2025,5(1):3.

Feng Cheng,Fu Xiongjun,Wang Ziyi,et al.An optimization method for collaborative radar antijamming based on multi-agent reinforcement learning [J].Remote Sensing,2023,15(11):2893.

Zhang Wenxu,Zhao Tong,Zhao Zhongkai,et al.An intelligent strategy decision method for collaborative jamming based on hierarchical multi-agent reinforcement learning [J]. IEEE Transactions on Cognitive Communications and Networking,2024,10(4):14671480.

黄洁瑜,谢军伟,杨子晴,等.雷达资源管理技术发展研究综述[J].现代雷达,2025,47(3):1-13.

Huang Jieyu,Xie Junwei,Yang Ziqing,et al.A review of radar resource management technology development research[J].Modern Radar,2025,47 (3):1-13.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

多源二维测角信息的三维动态目标跟踪定位算法

一般相关噪声下线性系统定区间平滑估计算法

一种基于角度测量估计飞行目标运动参数的新方法

利用角度测量估计机动目标运动参数的方法

采用动态载荷修正的汽轮发电机轴系动力学敏感性分析