基于强化学习的云资源混合式弹性伸缩算法

吴晓军; 张成; 原盛; 任晓春; 王玮

doi:10.7652/xjtuxb202201016

您当前的位置：

首页 >

文章列表页 >

基于强化学习的云资源混合式弹性伸缩算法

更新时间：2025-05-08

- 基于强化学习的云资源混合式弹性伸缩算法
- Blended Elastic Scaling Method for Cloud Resources Following Reinforcement Learning
- 西安交通大学学报 2022年56卷第1期页码：142-150
- 作者机构：
  
  西安交通大学软件学院,西安,710049
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202201016
  中图分类号： TP181
- 网络首发：2022-01-10，
  
  纸质出版：2022
- 稿件说明：
移动端阅览
吴晓军, 张成, 原盛, 等. 基于强化学习的云资源混合式弹性伸缩算法[J]. 西安交通大学学报, 2022,56(1):142-150.

Blended Elastic Scaling Method for Cloud Resources Following Reinforcement Learning[J]. 2022, 56(1): 142-150.
吴晓军, 张成, 原盛, 等. 基于强化学习的云资源混合式弹性伸缩算法[J]. 西安交通大学学报, 2022,56(1):142-150. DOI： 10.7652/xjtuxb202201016.

Blended Elastic Scaling Method for Cloud Resources Following Reinforcement Learning[J]. 2022, 56(1): 142-150. DOI： 10.7652/xjtuxb202201016.

摘要

为解决强化学习(RL)应用于云资源弹性伸缩问题时算法空间随应用规模变化而呈指数级变化、可伸缩性受限、算法训练时间长、收敛慢及设置水平伸缩动作空间时难以兼顾系统性能与稳定性等问题

提出了一种基于强化学习的分组云资源混合式伸缩算法(BGRL)。将应用实例进行逻辑分组

使算法空间固定

解决了算法空间爆炸及可伸缩性受限问题; 采用并行学习

加快了学习速度

解决了算法收敛慢的问题; 通过汇集多组的学习结果决定水平伸缩动作

解决了现有算法难以同时保证应用稳定性和资源调整及时性的问题; 采用水平和垂直两个方向上的混合式伸缩

在保证应用能力范围的同时

解决了局部性能问题。通过重放实际应用数据集而产生的工作负载模式进行云应用仿真

结果表明:BGRL的应用资源量最贴合负载变化

资源利用率最高

稳定在80%左右; 在消耗的资源量和违反服务质量请求的百分比方面

比其他算法分别减少了15%～20%和0.1%～3.26%。

Abstract

During applying reinforcement learning(RL)to the elastic scaling of cloud resources

we face these problems:the model space changing with application scale exponentially

limited scalability

long training time

slow convergence and difficulty in balancing the performance and stability of system as setting horizontal scaling action. A blended grouped scaling method of cloud resources based on reinforcement learning(BGRL)is proposed. The application examples are logically grouped to make the size of the model space fixed

which solves the problem of model space explosion versus limited algorithm scalability. The parallel learning method is adopted to speed up the model learning rate and solve the problem of slow algorithm convergence. By pooling the learning results of multiple groups to determine the horizontal scaling action

it solves the problem in the existing methods that are difficult to ensure the stability of application and the timeliness of resource adjustment at the same time. Blended expansion and contraction in both horizontal and vertical directions solve local performance problems while ensuring the scope of application capability. The cloud application simulation is performed by replaying the workload pattern generated by the actual application data set. The results show that the amount of application resources of BGRL is the most suitable for load changes

and the resource utilization rate reaches the highest

which remains stable at about 80%. Compared with the other methods

the percentage of requests violating the quality of service(QoS)is reduced by 15%-20% and 0.1%-3.26%

respectively.

关键词

Keywords

references

ZHANG Jian, YANG Yang, WANG Zhibo. Outsourcing large-scale systems of linear matrix equations in cloud computing [C]∥Proceedings of the 2016 IEEE 22nd International Conference on Parallel and Distributed Systems(ICPADS). Piscataway, NJ, USA: IEEE, 2016: 438-447.

龚强. 云计算关键技术之弹性伸缩控制技术认知研究 [J]. 信息技术, 2014, 38(1): 1-2, 6.

GONG Qiang. Research on the cognition of elastic retractable control technology: one of the key technology of cloud computing [J]. Information Technology, 2014, 38(1): 1-2, 6.

DEAN J, BARROSO L A. The tail at scale [J]. Communications of the ACM, 2013, 56(2): 74-80.

AL-DHURAIBI Y, PARAISO F, DJARALLAH N, et al. Elasticity in cloud computing: state of the art and research challenges [J]. IEEE Transactions on Services Computing, 2018, 11(2): 430-447.

SINGH P, GUPTA P, JYOTI K, et al. Research on auto-scaling of web applications in cloud: survey, trends and future directions [J]. Scalable Computing: Practice and Experience, 2019, 20(2): 399-432.

LORIDO-BOTRAN T, MIGUEL-ALONSO J, LOZANO J A. A review of auto-scaling techniques for elastic applications in cloud environments [J]. Journal of Grid Computing, 2014, 12(4): 559-592.

李茹杨, 彭慧民, 李仁刚, 等. 强化学习算法与应用综述 [J]. 计算机系统应用, 2020, 29(12): 13-25.

LI Ruyang, PENG Huimin, LI Rengang, et al. Overview on algorithms and applications for reinforcement learning [J]. Computer Systems Applications, 2020, 29(12): 13-25.

易鸣. 基于微服务架构应用平台的资源调度优化研究 [D]. 济南: 山东大学, 2020: 9-11.

王天泽. 基于灰色模型的云资源动态伸缩功能研究 [J]. 软件导刊, 2018, 17(4): 131-134.

WANG Tianze. Research on cloud resources auto-scaling based on grey model [J]. Software Guide, 2018, 17(4): 131-134.

尚小东, 张煜, 郭成昊. 基于作战任务优先级的容器云弹性伸缩系统 [J]. 指挥信息系统与技术, 2020, 11(3): 36-43.

SHANG Xiaodong, ZHANG Yu, GUO Chenghao. Container cloud elastic scaling system based on mission priority [J]. Command Information System and Technology, 2020, 11(3): 36-43.

SHEKHAR S, ABDEL-AZIZ H, BHATTACHARJEE A, et al. Performance interference-aware vertical elasticity for cloud-hosted latency-sensitive applications [C]∥Proceedings of the 2018 IEEE 11th International Conference on Cloud Computing(CLOUD). Piscataway, NJ, USA: IEEE, 2018: 82-89.

KARDANI-MOGHADDAM S, BUYYA R, RAMAMOHANARAO K. ADRL: a hybrid anomaly-aware deep reinforcement learning-based resource scaling in clouds [J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(3): 514-526.

GARÍ Y, MONGE D A, PACINI E, et al. Reinforcement learning-based application autoscaling in the cloud: a survey [J]. Engineering Applications of Artificial Intelligence, 2021, 102: 104288.

ASLANPOUR M S, GHOBAEI-ARANI M, NADJARAN TOOSI A. Auto-scaling web applications in clouds: a cost-aware approach [J]. Journal of Network and Computer Applications, 2017, 95: 26-41.

ALI-ELDIN A, TORDSSON J, ELMROTH E. An adaptive hybrid elasticity controller for cloud infrastructures [C]∥Proceedings of the 2012 IEEE Network Operations and Management Symposium. Piscataway, NJ, USA: IEEE, 2012: 204-212.

PATIKIRIKORALA T, COLMAN A. Feedback controllers in the cloud [C]∥Proceedings of the 2010 APSEC. Sydney, Australia: APSEC, 2010: 39.

GHOBAEI-ARANI M, JABBEHDARI S, POURMINA M A. An autonomic resource provisioning approach for service-based cloud applications: a hybrid approach [J]. Future Generation Computer Systems, 2018, 78: 191-210.

ARABNEJAD H, PAHL C, JAMSHIDI P, et al. A comparison of reinforcement learning techniques for fuzzy cloud auto-scaling [C]∥Proceedings of the 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing(CCGRID). Piscataway, NJ, USA: IEEE, 2017: 64-73.

HOROVITZ S, ARIAN Y. Efficient cloud auto-scaling with SLA objective using Q-learning [C]∥Proceedings of the 2018 IEEE 6th International Conference on Future Internet of Things and Cloud(FiCloud). Piscataway, NJ, USA: IEEE, 2018: 85-92.

YANG Zhe, NGUYEN P, JIN Haiming, et al. MIRAS: model-based reinforcement learning for microservice resource allocation over scientific workflows [C]∥Proceedings of the 2019 IEEE 39th International Conference on Distributed Computing Systems(ICDCS). Piscataway, NJ, USA: IEEE, 2019: 122-132.

ROSSI F, NARDELLI M, CARDELLINI V. Horizontal and vertical scaling of container-based applications using reinforcement learning [C]∥2019 IEEE 12th International Conference on Cloud Computing(CLOUD). Piscataway, NJ, USA: IEEE, 2019: 329-338.

GULISANO V, JERZAK Z, VOULGARIS S, et al. The DEBS 2016 grand challenge [C]∥Proceedings of the 10th ACM International Conference on Distributed and Event-based Systems. New York, USA: ACM, 2016: 289-292.

LI Yuqing, DAI Wenkuan, GAN Xiaoying, et al. Cooperative service placement and scheduling in edge clouds: a deadline-driven approach[J/OL]. IEEE Transactions on Mobile Computing, 2021 [2021-06-01]. https:∥ieeexplore.ieee.org/stamp/stamp.jsp?tp=arnumber=9361310.

YOU Wencong, JIAO Lei, LI Jun, et al. Scheduling DDoS cloud scrubbing in ISP networks via randomized online auctions [C]∥Proceedings of the IEEE Conference on Computer Communications. Piscataway, NJ, USA: IEEE, 2020: 1658-1667.

-------

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

应用深度强化学习的电池热管理系统控制策略

面向飞行器智能协同控制的分层双时延策略梯度强化学习方法

应用深度强化学习的电池热管理系统控制策略

联合遗传算法和强化学习的虚拟网络功能映射与调度方法

考虑驾驶风格的混合动力汽车强化学习能量管理策略