Stochastic Optimization Algorithms for Support Vector Machines Classification

Article type: Research Article

Authors: Bartkutė-Norkūnienė, Vaida

Affiliations: Institute of Mathematics and Informatics and Utena College, Akademijos 4, LT-08663 Vilnius, Lithuania, e-mail: vaidaba@ukolegija.lt

Abstract: In this paper, we consider the problem of semi-supervised binary classification by Support Vector Machines (SVM). This problem is explored as an unconstrained and non-smooth optimization task when part of the available data is unlabelled. We apply non-smooth optimization techniques to classification where the objective function considered is non-convex and non-differentiable and so difficult to minimize. We explore and compare the properties of Simulated Annealing and of Simultaneous Perturbation Stochastic Approximation (SPSA) algorithms (SPSA with the Lipschitz Perturbation Operator, SPSA with the Uniform Perturbation Operator, Standard Finite Difference Approximation) for semi-supervised SVM classification. Numerical results are given, obtained by running the proposed methods on several standard test problems drawn from the binary classification literature. The performance of the classifiers were evaluated by analyzing Receiver Operating Characteristics (ROC).

Keywords: support vector machine, semi-supervised classification, stochastic approximation, simulated annealing

Journal: Informatica, vol. 20, no. 2, pp. 173-186, 2009