A Parallel Service Function Chain Deployment Method Combining Federated Learning and Deep Reinforcement Learning

LIU Guangyuan; DU Jie; PANG Ziyuan

doi:10.7652/xjtuxb202509011

您当前的位置：

首页 >

文章列表页 >

A Parallel Service Function Chain Deployment Method Combining Federated Learning and Deep Reinforcement Learning

更新时间：2025-09-11

- A Parallel Service Function Chain Deployment Method Combining Federated Learning and Deep Reinforcement Learning
- Journal of Xi’an Jiaotong University Vol. 59, Issue 9, Pages: 110-121(2025)
- 作者机构：
  
  1.石家庄铁道大学信息科学与技术学院， 050043，石家庄
  2.河北省电磁环境效应与信息处理重点实验室， 050043，石家庄
  3.石家庄市人工智能重点实验室， 050043，石家庄
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202509011
  CLC： TP393.0
- Received：06 January 2025，
  
  Online First：20 June 2025，
  
  Published：10 September 2025
- 稿件说明：
移动端阅览
LIU Guangyuan, DU Jie, PANG Ziyuan. A Parallel Service Function Chain Deployment Method Combining Federated Learning and Deep Reinforcement Learning[J]. Journal of Xi’an Jiaotong University, 2025, 59(9): 110-121.
DOI：

LIU Guangyuan, DU Jie, PANG Ziyuan. A Parallel Service Function Chain Deployment Method Combining Federated Learning and Deep Reinforcement Learning[J]. Journal of Xi’an Jiaotong University, 2025, 59(9): 110-121. DOI： 10.7652/xjtuxb202509011.

摘要

针对多域边缘云网络中并行服务功能链（SFC）的动态部署问题，构建了一种优化的SFC并行结构，提出一种联合联邦学习（FedAvg）与深度强化学习（DRL）的新算法——FA-D3QN-PER。该方法解决了现有的单一DRL算法和集中式决策框架在解决SFC切分和部署时存在的资源分配不均和隐私泄露问题，通过允许各域内的智能体独立训练，并利用FedAvg共享模型参数，在保护数据隐私的同时优化全局策略。在部署阶段，对混合SFC的并行结构进行分析和优化；根据优化结果将优化后的混合SFC合理切分成若干个子链，并将其分配给合适的边缘域；将各子链中的虚拟网络功能（VNF）映射至目标域内的物理节点上。仿真结果表明，FA-D3QN-PER方法具有稳定性强、收敛速度快等特点，能够显著提高SFC部署的接受率，同时有效减少平均延迟和总成本，相较于FA-DQN、DFSC和MuL算法，FA-D3QN-PER算法将接受率提高了11.6%，平均延迟和总成本分别减少了17%和18.56%。

Abstract

To address the dynamic deployment of parallel service function chains (SFCs) in multi-domain edge cloud networks

this paper constructs an optimized parallel SFC architecture and proposes a novel approach combining federated learning (FedAvg) and deep reinforcement learning (DRL)

termed FA-D3QN-PER. This method resolves the issues of imbalanced resource allocation and privacy leakage inherent in existing single-agent DRL and centralized decision-making frameworks when handling SFC partitioning and deployment. By enabling independent training of agents within each domain and leveraging FedAvg for model parameter sharing

it optimizes global strategies while preserving data privacy. The deployment phase involves the analysis and optimization of the hybrid SFC parallel structure; intelligent partitioning of the optimized hybrid SFC into sub-chains and their allocation to suitable edge domains; mapping of virtual network functions (VNFs) within each sub-chain to physical nodes in target domains. Simulation results demonstrate that the FA-D3QN-PER method exhibits strong stability and fast convergence

significantly improving SFC deployment acceptance rates while effectively reducing average latency and total costs. Compared to FA-DQN

DFSC

and MuL

the FA-D3QN-PER method increases acceptance rates by 11.6%

while reducing average latency and total costs by 17% and 18.56%

respectively.

关键词

Keywords

references

HE Fujun , SATO T , OKI E . Optimization model for backup resource allocation in middleboxes with importance [J ] . IEEE/ACM Transactions on Networking , 2019 , 27 ( 4 ): 1742 - 1755 .

CAO Chen , WANG Jianhua , KWOK D , et al . webTWAS: a resource for disease candidate susceptibility genes identified by transcriptome-wide association study [J ] . Nucleic Acids Research , 2022 , 50 ( D1 ): D1123 - D1130 .

刘光远 , 曹晶仪 , 庞紫园 , 等 . 一种低时延虚拟网络功能映射及调度优化算法 [J ] . 西安交通大学学报 , 2023 , 57 ( 2 ): 121 - 130 .

LIU Guangyuan , CAO Jingyi , PANG Ziyuan , et al . An optimized algorithm with low latency for virtual network function mapping and scheduling [J ] . Journal of Xi'an Jiaotong University , 2023 , 57 ( 2 ): 121 - 130 .

孙士清 , 彭建华 , 游伟 , 等 . 5G网络下资源感知的服务功能链协同构建和映射算法 [J ] . 西安交通大学学报 , 2020 , 54 ( 8 ): 140 - 148 .

SUN Shiqing , PENG Jianhua , YOU Wei , et al . A coordinating composition and mapping algorithm for a service function chain with resource-aware [J ] . Journal of Xi'an Jiaotong University , 2020 , 54 ( 8 ): 140 - 148 .

KAUR K , MANGAT V , KUMAR K . A review on virtualized infrastructure managers with management and orchestration features in NFV architecture [J ] . Computer Networks , 2022 , 217 : 109281 .

XIE Sihao , MA Junte , ZHAO Jin . FlexChain: bridging parallelism and placement for service function chains [J ] . IEEE Transactions on Network and Service Management , 2021 , 18 ( 1 ): 195 - 208 .

AGARWAL S , CHINTAPALLI V R , TAMMA B R . FlexSFC: flexible resource allocation and VNF parallelism for improved SFC placement [C ] // 2022 IEEE 8th International Conference on Network Softwarization (NetSoft) . Piscataway, NJ, USA : IEEE , 2022 : 302 - 306 .

徐冉 , 王文东 , 龚向阳 , 等 . 网络功能虚拟化中延时感知的资源调度优化方法 [J ] . 计算机研究与发展 , 2018 , 55 ( 4 ): 738 - 747 .

XU Ran , WANG Wendong , GONG Xiangyang , et al . Delay-aware resource scheduling optimization in network function virtualization [J ] . Journal of Computer Research and Development , 2018 , 55 ( 4 ): 738 - 747 .

WANG Ying , ZHANG Leyi , YU Peng , et al . Reliability-oriented and resource-efficient service function chain construction and backup [J ] . IEEE Transactions on Network and Service Management , 2021 , 18 ( 1 ): 240 - 257 .

WU Yunyi , ZHENG Weichang , ZHANG Yongbing , et al . Reliability-aware VNF placement using a probability-based approach [J ] . IEEE Transactions on Network and Service Management , 2021 , 18 ( 3 ): 2478 - 2491 .

RUI Lanlan , CHEN Xushan , GAO Zhipeng , et al . Petri net-based reliability assessment and migration optimization strategy of SFC [J ] . IEEE Transactions on Network and Service Management , 2021 , 18 ( 1 ): 167 - 181 .

TOUMI N , BAGAA M , KSENTINI A . On using deep reinforcement learning for multi-domain SFC placement [C ] // 2021 IEEE Global Communications Conference (GLOBECOM) . Piscataway, NJ, USA : IEEE , 2021 : 1 - 6 .

JOSHI K D , KATAOKA K . pSMART: a lightweight, privacy-aware service function chain orchestration in multi-domain NFV/SDN [J ] . Computer Networks , 2020 , 178 : 107295 .

XIANG Qiao , WANG Xingchen , ZHANG Jianshun , et al . Unicorn: unified resource orchestration for multi-domain, geo-distributed data analytics [J ] . Future Generation Computer Systems , 2019 , 93 : 188 - 197 .

LIN Rongping , YU Song , LUO Shan , et al . Column generation based service function chaining embedding in multi-domain networks [J ] . IEEE Transactions on Cloud Computing , 2023 , 11 ( 1 ): 185 - 199 .

LIU Yi , ZHANG Hongqi , CHANG Dexian , et al . GDM: A general distributed method for cross-domain service function chain embedding [J ] . IEEE Transactions on Network and Service Management , 2020 , 17 ( 3 ): 1446 - 1459 .

HUANG Haojun , ZENG Cheng , ZHAO Yangmin , et al . Scalable orchestration of service function chains in NFV-enabled networks: a federated reinforcement learning approach [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 8 ): 2558 - 2571 .

CHEN Lixing , SHEN Cong , ZHOU Pan , et al . Collaborative service placement for edge computing in dense small cell networks [J ] . IEEE Transactions on Mobile Computing , 2021 , 20 ( 2 ): 377 - 390 .

WANG Jiadai , ZHAO Lei , LIU Jiajia , et al . Smart resource allocation for mobile edge computing: a deep reinforcement learning approach [J ] . IEEE Transactions on Emerging Topics in Computing , 2021 , 9 ( 3 ): 1529 - 1541 .

XIA Xiaoyu , CHEN Feifei , GRUNDY J , et al . Constrained app data caching over edge server graphs in edge computing environment [J ] . IEEE Transactions on Services Computing , 2022 , 15 ( 5 ): 2635 - 2647 .

SHANG Xiaojun , LIU Zhenhua , YANG Yuanyuan . Online service function chain placement for cost-effectiveness and network congestion control [J ] . IEEE Transactions on Computers , 2022 , 71 ( 1 ): 27 - 39 .

MAO Yingling , SHANG Xiaojun , YANG Yuanyuan . Provably efficient algorithms for traffic-sensitive SFC placement and flow routing [C ] // IEEE INFOCOM 2022-IEEE Conference on Computer Communications . Piscataway, NJ, USA : IEEE , 2022 : 950 - 959 .

PHAM C , TRAN N H , REN Shaolei , et al . Traffic-aware and energy-efficient vNF placement for service chaining: joint sampling and matching approach [J ] . IEEE Transactions on Services Computing , 2020 , 13 ( 1 ): 172 - 185 .

王珂 , 曲桦 , 赵季红 . 多域SFC部署中基于强化学习的多目标优化方法 [J ] . 计算机科学 , 2021 , 48 ( 12 ): 324 - 330 .

WANG Ke , QU Hua , ZHAO Jihong . Multi-objective optimization method based on reinforcement learning in multi-domain SFC deployment [J ] . Computer Science , 2021 , 48 ( 12 ): 324 - 330 .

KIBALYA G , SERRAT-FERNANDEZ J , GORRICHO J L , et al . A multi-stage graph aided algorithm for distributed service function chain provisioning across multiple domains [J ] . IEEE Access , 2021 , 9 : 114884 - 114904 .

CHEN Chen , NAGEL L , CUI Lin , et al . Distributed federated service chaining: a scalable and cost-aware approach for multi-domain networks [J ] . Computer Networks , 2022 , 212 : 109044 .

王琛 , 汤红波 , 游伟 , 等 . 一种5G网络低时延资源调度算法 [J ] . 西安交通大学学报 , 2018 , 52 ( 4 ): 117 - 124 .

WANG Chen , TANG Hongbo , YOU Wei , et al . A resource scheduling algorithm with low latency for 5G networks based on effective hybrid genetic algorithm and tabu search [J ] . Journal of Xi'an Jiaotong University , 2018 , 52 ( 4 ): 117 - 124 .

SUN Gang , LI Yayu , LIAO Dan , et al . Service function chain orchestration across multiple domains: a full mesh aggregation approach [J ] . IEEE Transactions on Network and Service Management , 2018 , 15 ( 3 ): 1175 - 1191 .

ZHU Yuchao , YAO Haipeng , MAI Tianle , et al . Multiagent reinforcement-learning-aided service function chain deployment for internet of things [J ] . IEEE Internet of Things Journal , 2022 , 9 ( 17 ): 15674 - 15684 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Intelligent Deformation Decision-Making Method for Morphing Glide Vehicle

Intelligent Deformation Decision-making Method for Morphing Glide Vehicle with Deep Reinforcement Learning

A Precise Control Method for Brain-Computer Cooperation with Deep Reinforcement Learning

Fuzzy Adaptive Deep Reinforcement Learning Method for Optimizing Transient Performance of Servo System

Deep Reinforcement Learning Control Strategy for Primary Frequency Regulation ofEnergy Storage Assisted Thermal Power Units

Related Author

CHEN Gang

MENG Fanyi

WANG Taojun

CHEN Gang

MENG Fanyi

WANG Taojun

张腾 1

蒋永玉 1

Related Institution

School of Aerospace Engineering, Xi'an Jiaotong University

School of Aerospace Engineering, Xi’an Jiaotong University

School of Mechanical Engineering, Xi’an Jiaotong University

School of Mechanical Engineering, Xi'an Jiaotong University

State Grid Henan Electric Power Company

⁰