优化极限学习机的序列最小优化方法

丁晓剑; 赵银亮

您当前的位置：

首页 >

文章列表页 >

优化极限学习机的序列最小优化方法

更新时间：2025-07-09

- 优化极限学习机的序列最小优化方法
- A Sequential Minimal Optimization Method for Optimization Extreme Learning Machine
- 西安交通大学学报 2011年45卷第6期页码：7-12+19
- 作者机构：
  
  西安交通大学电子与信息工程学院,西安,710049
- 作者简介：
- 基金信息：
  
  : 2010-09-09. 作者简介: 丁晓剑(1982-),男,博士生;赵银亮(联系人),男,教授,博士生导师. 基金项目: 国家高技术研究发展计划资助项目(2008AA01Z136)
- DOI：
  中图分类号： TP181
- 网络首发：2011-06-10，
  
  纸质出版：2011
- 稿件说明：
移动端阅览
丁晓剑, 赵银亮. 优化极限学习机的序列最小优化方法[J]. 西安交通大学学报, 2011,45(6):7-12+19.

A Sequential Minimal Optimization Method for Optimization Extreme Learning Machine[J]. 2011, 45(6): 7-12+19.
丁晓剑, 赵银亮. 优化极限学习机的序列最小优化方法[J]. 西安交通大学学报, 2011,45(6):7-12+19. DOI：

A Sequential Minimal Optimization Method for Optimization Extreme Learning Machine[J]. 2011, 45(6): 7-12+19. DOI：

摘要

针对传统二次规划求解方法训练优化极限学习机(OMELM)存在速度慢和效率低的问题

提出了单变量迭代序列最小优化(SSMO)算法.该算法通过在框式约束中优化拉格朗日乘子来实现目标函数的最小化:首先在初始化拉格朗日乘子中选择使目标函数值下降最大的拉格朗日乘子

将该拉格朗日乘子作为目标函数的唯一变量; 然后求解目标函数的最小值并更新该变量的值; 重复这个过程直到所有的拉格朗日乘子都满足二次规划问题的Karush-Kuhn-Tucker条件为止.实验结果表明:SSMO算法只需调节很少的参数值便可得到足够好的泛化性能; 采用SSMO算法的OMELM方法在泛化性能上要好于采用序列最小优化算法的支持向量机方法; 在随机数据集测试中

SSMO算法具有较好的鲁棒性.

Abstract

A sequential minimal optimization(SSMO)algorithm iterated with single variable is proposed to solve the slow speed and low efficiency problems in training optimization method based extreme learning machine(OMELM)by traditional quadratic programming solver. The algorithm searches the minimum value of the objective function by optimizing Lagrange multipliers within box constraints.The Lagrange multiplier that can generate maximum reduction in the objective function is selected from the initial Lagrange multipliers as for the unique variable of the objective function.Then the objective function is minimized with respect to the variable

and a new value of the Lagrange multiplier is obtained. The process is repeated until all Lagrange multipliers satisfy the Karush-Kuhn-Tucker condition of the quadratic programming problem.Experimental results show that SSMO algorithm can obtain good enough generalization performance with adjusting few parameter values.The generalization performance of OMELM method using SSMO algorithm is better than that of the support vector machine(SVM)method using sequential minimal optimization(SMO)algorithm.SSMO algorithm is robust in random datasets trails.

关键词

Keywords

references

HUANG Guangbin, ZHU Qinyu, SIEW C K. Extreme learning machine: theory and applications [J]. Neurocomputing, 2006, 70(1/2/3): 489-501.

HUANG Guangbin, ZHU Qinyu, SIEW C K. Real-time learning capability of neural networks [J]. IEEE Transactions on Neural Network, 2006, 17(2): 863-878.

程松, 闫建伟, 赵登福, 等. 短期负荷预测的集成改进极端学习机方法[J]. 西安交通大学学报, 2009, 43(2): 106-110.

CHENG Song, YAN Jianwei, ZHAO Dengfu, et al. Short-term load forecasting method based on ensemble improved extreme learning machine [J]. Journal of Xi'an Jiaotong University, 2009, 43(2): 106-110.

邓万宇, 郑庆华, 陈琳, 等. 神经网络极速学习方法研究[J]. 计算机学报, 2010, 33(2): 279-287.

DENG Wanyu, ZHENG Qinghua, CHEN Lin, et al. Research on extreme learning of neural networks [J]. Chinese Journal of Computers, 2010, 33(2): 279-287.

BARTLETT P L. The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network [J]. IEEE Transactions on Information Theory, 1998, 44(2): 525-536.

HUANG Guangbin, DING Xiaojian, ZHOU Hongming. Optimization method based extreme learning machine for classification [J]. Neurocomputing, 2010, 74(1/2/3): 155-163.

PLATT J. Fast training of support vector machines using sequential minimal optimization [M]∥Advances in Kernel Methods: Support Vector Learning. Cambridge, MA, USA: MIT Press, 1999: 185-208.

FLETCHER R. Practical methods of optimization: constrained optimization [M]. New York, USA: John Wiley and Sons, 1981: 2.

BLAKE C L, MERZ C J. UCI repository of machine learning databases [EB/OL].(1998-04-02)[2010-02-12]. http:∥www.ics.uci.edu/～mlearn/MLRepository.html.

MICHIE D, SPIEGELHALTER D J, TAYLOR C C. Machine learning, neural and statistical classification [M]. Englewood Cliffs, NJ, USA: Prentice Hall, 1994.

GHAUTY P, PAUL S, PAL N R. NEUROSVM: an architecture to reduce the effect of the choice of kernel on the performance of SVM [J]. Journal of Machine Learning Research, 2009, 10(3):591-622.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

高超声速飞行器干扰补偿与解耦最优反步控制策略

面向分类去噪问题的模糊支持向量机新算法

一种新的浮选泡沫图像识别方法

泛化的统一切比雪夫多项式核函数

基于样本熵的注意力相关脑电特征信息提取与分类