A kernel-based Adaline for function approximation

Article type: Research Article

Authors: Frieß, T-T.^{; *} | Harrison, R.F.

Affiliations: University of Sheffield, Department of Automatic Control and Systems Engineering, Mappin Street, S1 3JD, England, UK

Correspondence: [*] Corresponding author. E-mail address: friess@acse.shef.ac.uk (T-T. Frieß)

Abstract: In this work the kernel Adaline algorithm is presented. The new algorithm is a generalisation of Widrow's and Hoff's linear Adaline, and allows to approximate non-linear functional relationships. Similar to the linear adaline, the proposed neural network algorithm minimises the least-mean-squared (LMS) cost function. It can be guaranteed that the kernel Adaline's cost function is always convex, therefore the method does not suffer from local optima as known in conventional neural networks. The algorithm uses potential function operators due to Aizerman and colleagues to map the training points in a first stage into a very high dimensional non-linear “feature” space. In the second stage the LMS solution in this space is determined by the algorithm. Weight decay regularisation allows to avoid overfitting effects, and can be performed efficiently. The kernel Adaline algorithm works in a sequential fashion, is conceptually simple, and numerically robust. The method shows a high performace in tasks like one dimensional curve fitting, system identification, and speech processing.

Keywords: Adaline, Method of potential functions, Perceptron, Neural networks

DOI: 10.3233/IDA-1999-3405

Journal: Intelligent Data Analysis, vol. 3, no. 4, pp. 307-313, 1999