Resource Allocation and Dynamic Deployment Algorithm for Unmanned Aerial Vehicle Enabled Base Stations in Air-Ground Networks

ZHANG Shangwei; HE Simeng

doi:10.7652/xjtuxb202403016

您当前的位置：

首页 >

文章列表页 >

Resource Allocation and Dynamic Deployment Algorithm for Unmanned Aerial Vehicle Enabled Base Stations in Air-Ground Networks

更新时间：2025-07-09

- Resource Allocation and Dynamic Deployment Algorithm for Unmanned Aerial Vehicle Enabled Base Stations in Air-Ground Networks
- Vol. 58, Issue 3, Pages: 172-182(2024)
- 作者机构：
  
  西北工业大学网络空间安全学院,西安,710072
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202403016
  CLC： TN929.5
- Online First：10 March 2024，
  
  Published：2024
- 稿件说明：
移动端阅览
ZHANG Shangwei, HE Simeng. Resource Allocation and Dynamic Deployment Algorithm for Unmanned Aerial Vehicle Enabled Base Stations in Air-Ground Networks[J]. 2024, 58(3): 172-182.
DOI：

ZHANG Shangwei, HE Simeng. Resource Allocation and Dynamic Deployment Algorithm for Unmanned Aerial Vehicle Enabled Base Stations in Air-Ground Networks[J]. 2024, 58(3): 172-182. DOI： 10.7652/xjtuxb202403016.

摘要

为解决空地网络中地面设备数量变化引起的用户体验质量无法满足的问题

提出了一种智能网络资源分配与多无人机基站动态部署方案。首先

考虑用户体验质量和无人机、地面设备能量约束

以最小化系统总能耗为目标进行问题建模; 其次

将多无人机的动态部署问题转化成具有连续动作集的马尔可夫决策过程

并根据优化目标设计了基于能耗惩罚的奖励函数; 然后

采用基于确定性策略梯度的深度强化学习算法求解此问题; 最后

通过仿真和对比实验验证所提方案的有效性和优越性。实验结果表明:对于海量用户场景

所提算法比深度强化学习和演员-评论家算法有更好的收敛性和更高的累积奖励

与单无人机和传统地面基站部署方案相比

所提方案系统的能耗降低约30%～40%

用户服务质量满意度提升约50%～60%。

Abstract

To address the problem of unsatisfactory user experience quality caused by the fluctuation of ground device quantity in an air-ground network

a solution for intelligent network resource allocation and dynamic deployment of base stations with multiple unmanned aerial vehicles(UAVs)is proposed. Firstly

considering user experience quality and the energy constraints of UAVs and ground devices

the problem is modeled with the objective of minimizing the total system energy consumption. Secondly

the dynamic deployment of multiple UAVs is transformed into a Markov decision process(MDP)with a continuous action set

and a reward function based on energy penalty is designed according to the optimization objective. Thirdly

a deep reinforcement learning algorithm based on deep deterministic policy gradient(DDPG)is used to solve this problem. Lastly

the effectiveness and superiority of the proposed solution are verified through simulation and comparative experiment. Experimental results show that

for scenarios with a massive number of users

the proposed algorithm exhibits better convergence and higher cumulative rewards compared to deep reinforcement learning and actor-critic algorithms. In comparison to single UAV and traditional ground base station deployment solutions

the proposed solution reduces energy consumption by approximately 30% to 40%

and improves user satisfaction with service quality by around 50% to 60%.

关键词

Keywords

references

陈新颖, 盛敏, 李博, 等. 面向6G的无人机通信综述 [J]. 电子与信息学报, 2022, 44(3): 781-789.

CHEN Xinying, SHENG Min, LI Bo, et al. Survey on unmanned aerial vehicle communications for 6G [J]. Journal of Electronics & Information Technology, 2022, 44(3): 781-789.

薛德鑫, 单涛, 徐宁骏, 等. 军用作战无人机未来发展研究 [J]. 指挥控制与仿真, 2022, 44(6): 1-6.

XUE Dexin, SHAN Tao, XU Ningjun, et al. Research on the future development of military combat UAV [J]. Command Control Simulation,2022, 44(6): 1-6.

ZHANG Shangwei, LIU Jiajia, ZHU Yajie, et al. Joint computation offloading and trajectory design for aerial computing [J]. IEEE Wireless Communications, 2021, 28(5): 88-94.

WANG Liang, WANG Kezhi, PAN Cunhua, et al. Multi-agent deep reinforcement learning-based trajectory planning for multi-UAV assisted mobile edge computing [J]. IEEE Transactions on Cognitive Communications and Networking, 2021, 7(1): 73-84.

SHEN Lingfeng, WANG Ning, ZHANG Di, et al. Energy-aware dynamic trajectory planning for UAV-enabled data collection in mMTC networks [J]. IEEE Transactions on Green Communications and Networking, 2022, 6(4): 1957-1971.

ZHU Botao, BEDEER E, NGUYEN H H, et al. UAV trajectory planning in wireless sensor networks for energy consumption minimization by deep reinforcement learning [J]. IEEE Transactions on Vehicular Technology, 2021, 70(9): 9540-9554.

TAN Zhenjie, QU Hua, ZHAO Jihong, et al. UAV-aided edge/fog computing in smart IoT community for social augmented reality [J]. IEEE Internet of Things Journal, 2020, 7(6): 4872-4884.

ZHAN Cheng, HU Han, SUI Xiufeng, et al. Completion time and energy optimization in the UAV-enabled mobile-edge computing system [J]. IEEE Internet of Things Journal, 2020, 7(8): 7808-7822.

HUANG Zhengrui, CHEN Chongcheng, PAN Miaoxin. Multiobjective UAV path planning for emergency information collection and transmission [J]. IEEE Internet of Things Journal, 2020, 7(8): 6993-7009.

ZHU Lipeng, ZHANG Jun, XIAO Zhenyu, et al. 3-D beamforming for flexible coverage in millimeter-wave UAV communications [J]. IEEE Wireless Communications Letters, 2019, 8(3): 837-840.

CHENG Nan, WU Shen, WANG Xiucheng, et al. AI for UAV-assisted IoT applications: a comprehensive review [J]. IEEE Internet of Things Journal, 2023, 10(16): 14438-14461.

LIU Yi, XIE Shengli, ZHANG Yan. Cooperative offloading and resource management for UAV-enabled mobile edge computing in power IoT system [J]. IEEE Transactions on Vehicular Technology, 2020, 69(10): 12229-12239.

夏景明, 刘玉风, 谈玲. 基于蜂窝网络的多无人机能量消耗最优化算法研究 [J]. 通信学报, 2023, 44(2): 185-197.

XIA Jingming, LIU Yufeng, TAN Ling. Research on multi-UAV energy consumption optimization algorithm for cellular-connected network [J]. Journal on Communications, 2023, 44(2): 185-197.

BOR-YALINIZ R I, EL-KEYI A, YANIKOMEROGLU H. Efficient 3-D placement of an aerial base station in next generation cellular networks [C]//2016 IEEE International Conference on Communications(ICC). Piscataway, NJ, USA: IEEE, 2016: 1-5.

BITHAS P S, NIKOLAIDIS V, KANATAS A G, et al. UAV-to-ground communications: channel modeling and UAV selection [J]. IEEE Transactions on Communications, 2020, 68(8): 5135-5144.

CELIK A, RADAYDEH R M, AL-QAHTANI F S, et al. Resource allocation and cluster formation for imperfect NOMA in DL/UL decoupled HetNets [C]//2017 IEEE Globecom Workshops(GC Wkshps). Piscataway, NJ, USA: IEEE, 2017: 1-6.

NA Zhenyu, LIU Yue, SHI Jingcheng, et al. UAV-supported clustered NOMA for 6G-enabled internet of things: trajectory planning and resource allocation [J]. IEEE Internet of Things Journal, 2021, 8(20): 15041-15048.

SHINKUMA R, KISHI N, OTA K, et al. Smarter base station sleeping for greener cellular networks [J]. IEEE Network, 2021, 35(6): 98-103.

ANDREAE J H. STELLA: a scheme for a learning machine [J]. IFAC Proceedings Volumes, 1963, 1(2): 497-502.

WANG Yunpeng, FANG Weiwei, DING Yi, et al. Computation offloading optimization for UAV-assisted mobile edge computing: a deep deterministic policy gradient approach [J]. Wireless Networks, 2021, 27(4): 2991-3006.

FAN Jianqing, WANG Zhaoran, XIE Yuchen, et al. A theoretical analysis of deep Q-learning [C]//Proceedings of the 2nd Conference on Learning for Dynamics and Control. Chia Laguna Resort, Sardinia, Italy: PMLR, 2020: 486-489.

TAN Fuxiao, YAN Pengfei, GUAN Xinping. Deep reinforcement learning: from Q-learning to deep Q-learning [C]//Proceedings of the 24th International Conference on Neural Information Processing. Cham: Springer International Publishing, 2017: 475-483.

LIU Qian, SHI Long, SUN Linlin, et al. Path planning for UAV-mounted mobile edge computing with deep reinforcement learning [J]. IEEE Transactions on Vehicular Technology, 2020, 69(5): 5723-5728.

WANG Liang, WANG Kezhi, PAN Cunhua, et al. Deep reinforcement learning based dynamic trajectory control for UAV-assisted mobile edge computing [J]. IEEE Transactions on Mobile Computing, 2022, 21(10): 3536-3550.

ZHANG Shangwei, WANG Xiao, SHI Zhenjiang, et al. Reinforcement learning based RSS-threshold optimization for D2D-Aided HTC/MTC in dense NOMA systems [J]. IEEE Transactions on Wireless Communications, 2023, 22(10): 6489-6503.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Parallel Service Function Chain Deployment Method Combining Federated Learning and Deep Reinforcement Learning

A Long-Term Unmanned Aerial Vehicle Tracking Algorithm Using Local-Global RegionRedetection Mechanism

A Resource Allocation Scheme Using Graph Theory for D2D Communication in Multi-Cell Heterogeneous Cellular Networks

Two Mode Selection and Resource Allocation Schedules for Device-to-Device Communication with Mobile Relay Assistance

A Resource Allocation Method for Clusters of Streaming Media Servers in Multi-Version VoD

Related Author

PANG Ziyuan

DU Jie

LIU Guangyuan

HUANG He

MA Haoran

LIU Guoquan

WANG Huifeng

GAO Tao

Related Institution

School of Information Science and Technology, Shijiazhuang Tiedao University

Hebei Provincial Key Laboratory of Electromagnetic Environmental Effects and Information Processing

Shijiazhuang Key Laboratory of Artificial Intelligence

Xi'an Key Laboratory of Intelligent Expressway Information Fusion and Control, Chang'an University

School of Electronic and Control Engineering, Chang'an University

⁰