Affiliations: [a] College of Mathematics and Information Science, Hebei University, Baoding, Hebei, China | [b] Key Laboratory of Machine Learning and Computational Intelligence of Hebei Province, Baoding, Hebei, China | [c] College of Computer Science and Software, Shenzhen University, Shenzhen, China
Correspondence:
[*]
Corresponding author: Shuxia Lu, College of Mathematics and Information Science, Hebei University, Baoding, Hebei 071002, China. E-mail:cmclusx@126.com
Abstract: Extreme learning machine (ELM) is a learning algorithm
for single-hidden layer feedforward neural networks (SLFNs) which randomly
chooses hidden nodes and analytically determines the output weights of
SLFNs. After the input weights and the hidden layer biases are chosen
randomly, ELM can be simply considered a linear system. However, the
learning time of ELM is mainly spent on calculating the Moore-Penrose
inverse matrices of the hidden layer output matrix. This paper focuses on
effective computation of the Moore-Penrose inverse matrices for ELM, several
methods are proposed. They are the reduced QR factorization with column
Pivoting and Geninv ELM (QRGeninv-ELM), tensor product matrix ELM (TPM-ELM).
And we compare QRGeninv-ELM, TPM-ELM with the relational algorithm of
Moore-Penrose inverse matrices for ELM, the relational algorithms are:
Cholesky factorization of singular matrix ELM (Geninv-ELM), QR factorization
and Ginv ELM (QRGinv-ELM), the conjugate Gram-Schmidt process ELM (CGS-ELM).
The experimental results and the statistical analysis of the experimental
results both demonstrate that QRGeninv-ELM, TPM-ELM and Geninv-ELM are
faster than other kinds of ELM and can reach comparable generalization
performance.
Keywords: Extreme learning machine, tensor product matrix, cholesky factorization of singular matrix, conjugate gram-schmidt process, QR factorization
