A smooth extreme learning machine framework

Article type: Research Article

Authors: Yang, Liming^{; *} | Zhang, Siyun

Affiliations: College of Science, China Agricultural University, Beijing, China

Correspondence: [*] Corresponding author. Liming Yang, College of Science, China Agricultural University, Beijing 100083, China. E-mail: cauyanglm@163.com.

Abstract: Extreme learning machine (ELM) has demonstrated great potential in machine learning and data mining. Smoothing strategy is an important technology for continuous optimizations. In this work, we apply a smoothing technique to replace the hinge loss function by an accurate smooth approximation. This will allow us to solve ELM as an unconstrained minimization problem directly. We term this reformulated problem as smooth ELM (SELM). A Newton-Armijo algorithm is used to solve the proposed SELM, and the resulting algorithm converges globally and quadratically. The proposed SELM with fast running speed has less decision variables and can better deal with nonlinear problems than the existing smooth support vector machine. Numerical experiments on various types of datasets including two-class datasets and multi-class datasets demonstrate that the speed of SELM is much faster than that of the existing ELM models. And compared with other popular algorithms of support vector machine and ELM, the proposed SELM achieves better or similar generalization. These demonstrate the effectiveness and fast speed of the algorithm.

Keywords: Extreme learning machine, neural networks, smoothing technique, Newton-Armijo algorithm

DOI: 10.3233/JIFS-162162

Journal: Journal of Intelligent & Fuzzy Systems, vol. 33, no. 6, pp. 3373-3381, 2017