Setting attribute weights for k-NN based binary classification via quadratic programming

Article type: Research Article

Authors: Zhang, Lu^{; *} | Coenen, Frans | Leng, Paul

Affiliations: Department of Computer Science, University of Liverpool, Liverpool L69 3BX, UK. Tel.: +44 151 7943792; Fax: +44 151 7943715; E-mail: lzhang@csc.liv.ac.uk, frans@csc.liv.ac.uk, phl@csc.liv.ac.uk

Correspondence: [*] Corresponding author.

Abstract: The k-Nearest Neighbour (k-NN) method is a typical lazy learning paradigm for solving classification problems. Although this method was originally proposed as a non-parameterised method, attribute weight setting has been commonly adopted to deal with irrelevant attributes. In this paper, we propose a new attribute weight setting method for k-NN based classifiers using quadratic programming, which is particularly suitable for binary classification problems. Our method formalises the attribute weight setting problem as a quadratic programming problem and exploits commercial software to calculate attribute weights. To evaluate our method, we carried out a series of experiments on six established data sets. Experiments show that our method is quite practical for various problems and can achieve a stable increase in accuracy over the standard k-NN method as well as a competitive performance. Another merit of the method is that it can use small training sets.

Keywords: classification, machine learning, k-Nearest Neighbour, attribute weight, quadratic programming

DOI: 10.3233/IDA-2003-7504

Journal: Intelligent Data Analysis, vol. 7, no. 5, pp. 427-441, 2003