On optimal degree selection for polynomial kernel with support vector machines: Theoretical and empirical investigations

Article type: Research Article

Authors: Ali, Shawkat^{a; *} | Smith-Miles, Kate^b

Affiliations: [a] School of Computer Science, Central Queensland University, QLD 4702, Australia VIC 3125, Australia | [b] School of Engineering and Information Technology, Deakin University, QLD 4702, Australia VIC 3125, Australia. E-mail: kate.smith-miles@deakin.edu.au

Correspondence: [*] Corresponding author. E-mail: s.ali@cqu.edu.au

Abstract: The key challenge in kernel based learning algorithms is the choice of an appropriate kernel and its optimal parameters. Selecting the optimal degree of a polynomial kernel is critical to ensure good generalisation of the resulting support vector machine model. In this paper we propose Bayesian and Laplace approximation methods to estimate the polynomial degree. A rule based meta-learning approach is then proposed for automatic polynomial kernel and its optimal degree selection. The new approach is constructed and tested on different sizes of 112 datasets with binary class as well as multi class classification problems. An extensive computational evaluation of these methods is conducted, and rules are generated to determine when these approximation methods are appropriate.

Keywords: Support vector machines, polynomial kernel, rule based method

DOI: 10.3233/KES-2007-11101

Journal: International Journal of Knowledge-based and Intelligent Engineering Systems, vol. 11, no. 1, pp. 1-18, 2007