联合遗传算法和强化学习的虚拟网络功能映射与调度方法

刘光远; 曹晶仪; 杜婕

doi:10.7652/xjtuxb202408018

您当前的位置：

首页 >

文章列表页 >

联合遗传算法和强化学习的虚拟网络功能映射与调度方法

—————————————————————— | 更新时间：2025-07-28

- 联合遗传算法和强化学习的虚拟网络功能映射与调度方法
- Function Mapping and Scheduling Method of Virtual Network Combining Genetic Algorithm and Reinforcement Learning
- 西安交通大学学报 2024年58卷第8期页码：175-184
- 作者机构：
  
  1. 石家庄铁道大学信息科学与技术学院,石家庄,050043
  2. 河北省电磁环境效应与信息处理重点实验室,石家庄,050043
- 作者简介：
  
  : 2023-10-06。作者简介: 刘光远(1981—),男,副教授,硕士生导师。基金项目: 国家自然科学基金资助项目(62106157)
- 基金信息：
  
  : 2023-10-06。作者简介: 刘光远(1981—),男,副教授,硕士生导师。基金项目: 国家自然科学基金资助项目(62106157);国家重点研发计划资助项目(2018YFB1701403)。
- DOI：10.7652/xjtuxb202408018
  中图分类号： TP393
- 纸质出版：2024
- 稿件说明：
移动端阅览
刘光远, 曹晶仪, 杜婕. 联合遗传算法和强化学习的虚拟网络功能映射与调度方法[J]. 西安交通大学学报, 2024,58(8):175-184.

LIU Guangyuan, CAO Jingyi, DU Jie. Function Mapping and Scheduling Method of Virtual Network Combining Genetic Algorithm and Reinforcement Learning[J]. 2024, 58(8): 175-184.
刘光远, 曹晶仪, 杜婕. 联合遗传算法和强化学习的虚拟网络功能映射与调度方法[J]. 西安交通大学学报, 2024,58(8):175-184. DOI： 10.7652/xjtuxb202408018.

LIU Guangyuan, CAO Jingyi, DU Jie. Function Mapping and Scheduling Method of Virtual Network Combining Genetic Algorithm and Reinforcement Learning[J]. 2024, 58(8): 175-184. DOI： 10.7652/xjtuxb202408018.

摘要

网络功能虚拟化环境下

为满足用户不同需求并提高资源利用效率

将虚拟网络功能映射与调度联合考虑。首先

通过构建网络低时延、虚拟机资源高利用及能量低损耗的多目标优化模型

设计了一种强化学习(RL)联合第三代非支配排序遗传算法(NSGA Ⅲ)的优化方法RL-NSGA Ⅲ; 然后

采用两段式初始化技术求得高质量初始解

利用强化学习的优势自适应调节交叉变异参数

以保持种群多样性; 最后

基于参考点的第三代非支配排序遗传算法

将虚拟网络功能映射至虚拟机并进行调度服务

得到多目标优化策略。仿真结果表明:相较于已有的NSGA Ⅲ、NSGA Ⅱ和MOPSO算法

采用RL-NSGA Ⅲ算法计算得到的时延降低了17%～28%

节点负荷提高了9%～19%

能量损耗降低了12%～26%

表明所提算法在提高网络速率和降低网络运营支出上的有效性。

Abstract

In order to meet different needs of users and improve resource utilization efficiency in the network function virtualization environment

this paper examines virtual network function mapping and scheduling together. Firstly

an optimization method RL-NSGA Ⅲ based on reinforcement learning(RL)combined with the third-generation non-dominated sorting genetic algorithm(NSGA Ⅲ)is designed by constructing a multi-objective optimization model with low network delay

high utilization of virtual machine resources and low energy loss. Then

the two-stage initialization technique is used to obtain a high-quality initial solution

and RL's advantage is used to adjust the crossover and variation parameters adaptively to maintain the diversity of the population. Finally

NSGA Ⅲ based on reference points is used to map virtual network functions for virtual machines and perform scheduling services to obtain a multi-objective optimization strategy. The simulation results show that compared with the existing methods NSGA Ⅲ

NSGA Ⅱ and MOPSO

the RL-NSGA Ⅲ method reduces latency by 17% to 28%

improves node load by 9% to 19%

and reduces energy loss by 12% to 26%

which verifies the effectiveness of the proposed method in improving network speed and reducing network operating expenses.

关键词

Keywords

references

LI Biyi, CHENG Bo, LIU Xuan, et al. Joint resource optimization and delay-aware virtual network function migration in data center networks [J]. IEEE Transactions on Network and Service Management, 2021, 18(3): 2960-2974.

ZHANG Dong, ZHENG Zhifan, LIN Xiang, et al. Dynamic backup sharing scheme of service function chains in NFV [J]. China Communications, 2022, 19(5): 178-190.

GIL HERRERA J D J, BOTERO VEGA J F. Network functions virtualization: a survey [J]. IEEE Latin America Transactions, 2016, 14(2): 983-997.

孙士清, 彭建华, 游伟, 等. 5G网络下资源感知的服务功能链协同构建和映射算法 [J]. 西安交通大学学报, 2020, 54(8): 140-148.

SUN Shiqing, PENG Jianhua, YOU Wei, et al. A coordinating composition and mapping algorithm for a service function chain with resource-aware [J]. Journal of Xi'an Jiaotong University, 2020, 54(8): 140-148.

王珂, 曲桦, 赵季红. 多域SFC部署中基于强化学习的多目标优化方法 [J]. 计算机科学, 2021, 48(12): 324-330.

WANG Ke, QU Hua, ZHAO Jihong. Multi-objective optimization method based on reinforcement learning in multi-domain SFC deployment [J]. Computer Science, 2021, 48(12): 324-330.

唐伦, 贺兰钦, 连沁怡, 等. 基于改进深度强化学习的虚拟网络功能部署优化算法 [J]. 电子与信息学报, 2021, 43(6): 1724-1732.

TANG Lun, HE Lanqin, LIAN Qinyi, et al. Virtual network function placement optimization algorithm based on improve deep reinforcement learning [J]. Journal of Electronics Information Technology, 2021, 43(6): 1724-1732.

刘光远, 曹晶仪, 庞紫园, 等. 一种低时延虚拟网络功能映射及调度优化算法 [J]. 西安交通大学学报, 2023, 57(2): 121-130.

LIU Guangyuan, CAO Jingyi, PANG Ziyuan, et al. An optimized algorithm with low latency for virtual network function mapping and scheduling [J]. Journal of Xi'an Jiaotong University, 2023, 57(2): 121-130.

徐冉, 王文东, 龚向阳, 等. 网络功能虚拟化中延时感知的资源调度优化方法 [J]. 计算机研究与发展, 2018, 55(4): 738-747.

XU Ran, WANG Wendong, GONG Xiangyang, et al. Delay-aware resource scheduling optimization in network function virtualization [J]. Journal of Computer Research and Development, 2018, 55(4): 738-747.

魏德宾, 杨鹏, 杨力, 等. 一种基于卫星网络的虚拟网络功能快速映射算法 [J]. 计算机科学, 2020, 47(3): 248-254.

WEI Debin, YANG Peng, YANG Li, et al. Virtual network function fast mapping algorithm over satellite network [J]. Computer Science, 2020, 47(3): 248-254.

TAVAKOLI-SOMEH S, REZVANI M H. Multi-objective virtual network function placement using NSGA-Ⅱ meta-heuristic approach [J]. The Journal of Supercomputing, 2019, 75(10): 6451-6487.

TARIQ M A, FAROOQ M U, ZEESHAN M, et al. FIPAM: fuzzy inference based placement and migration approach for NFV-Based IoTs [J]. IEEE Transactions on Network and Service Management, 2022, 19(4): 4298-4309.

朱国晖, 刘璐, 雷兰洁. 基于VNF组合的服务功能链设计及映射算法 [J]. 计算机工程, 2020, 46(4): 183-188, 197.

ZHU Guohui, LIU Lu, LEI Lanjie. Service function chain design and mapping algorithm based on VNF combination [J]. Computer Engineering, 2020, 46(4): 183-188, 197.

YUAN Quan, TANG Hongbo, YOU Wei, et al. Virtual network function scheduling via multilayer encoding genetic algorithm with distributed bandwidth allocation [J]. Science China Information Sciences, 2018, 61(9): 092107.

LI Qi, WANG Xing, ZHAO Tao, et al. An improved genetic algorithm for the scheduling of virtual network functions [C]//2019 20th Asia-Pacific Network Operations and Management Symposium(APNOMS). Piscataway, NJ, USA: IEEE, 2019: 1-4.

JIA Junzhong, YANG Lei, CAO Jiannong. Reliability-aware dynamic service chain scheduling in 5G networks based on reinforcement learning [C]//IEEE INFOCOM 2021-IEEE Conference on Computer Communications. Piscataway, NJ, USA: IEEE, 2021: 1-10.

PRO MWONGSA N, EBRAHIMZADEH A, GLITHO R H, et al. Joint VNF placement and scheduling for latency-sensitive services [J]. IEEE Transactions on Network Science and Engineering, 2022, 9(4): 2432-2449.

KUAI Zhenran, WANG Tianyu, WANG Shaowei. Fair virtual network function mapping and scheduling using proximal policy optimization [J]. IEEE Transactions on Communications, 2022, 70(11): 7434-7445.

HU Hongchao, ZHANG Fan, MAO Yuxing, et al. A forwarding graph embedding algorithm exploiting regional topology information [J]. Frontiers of Information Technology Electronic Engineering, 2017, 18(11): 1854-1866.

DEB K, JAIN H. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach: solving problems with box constraints [J]. IEEE Transactions on Evolutionary Computation, 2014, 18(4): 577-601.

CHEN Ronghua, YANG Bo, LI Shi, et al. A self-learning genetic algorithm based on reinforcement learning for flexible job-shop scheduling problem [J]. Computers Industrial Engineering, 2020, 149: 106778.

张韵, 钟慧超, 张春江, 等. 基于机器学习的多策略并行遗传算法 [J]. 计算机集成制造系统, 2021, 27(10): 2921-2928.

ZHANG Yun, ZHONG Huichao, ZHANG Chunjiang, et al. Multi-strategy parallel genetic algorithm based on machine learning [J]. Computer Integrated Manufacturing Systems, 2021, 27(10): 2921-2928.

王琛, 汤红波, 游伟, 等. 一种5G网络低时延资源调度算法 [J]. 西安交通大学学报, 2018, 52(4): 117-124.

WANG Chen, TANG Hongbo, YOU Wei, et al. A resource scheduling algorithm with low latency for 5G networks based on effective hybrid genetic algorithm and Tabu search [J]. Journal of Xi'an Jiaotong University, 2018, 52(4): 117-124.

GAMAL M, JAFARIZADEH S, ABOLHASAN M, et al. Mapping and scheduling for non-uniform arrival of virtual network function(VNF)requests [C]//2019 IEEE 90th Vehicular Technology Conference(VTC2019-Fall). Piscataway, NJ, USA: IEEE, 2019: 1-6.

CHANTRE H D, DA FONSECA N L S. Multi-objective optimization for edge device placement and reliable broadcasting in 5G NFV-based small cell networks [J]. IEEE Journal on Selected Areas in Communications, 2018, 36(10): 2304-2317.

WANG Ranyin, AGHVAMI H, FRIDERIKOS V. An end-to-end network slicing design policy [C]//2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications. Piscataway, NJ, USA: IEEE, 2020: 1-6.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

一种低时延虚拟网络功能映射及调度优化算法

应用深度强化学习的电池热管理系统控制策略

面向飞行器智能协同控制的分层双时延策略梯度强化学习方法

应用深度强化学习的电池热管理系统控制策略

考虑驾驶风格的混合动力汽车强化学习能量管理策略