On the effectiveness of Parzen window classifier

Article type: Research Article

Authors: Raudys, Šarūnas

Affiliations: Institute of Mathematics and Informatics, Lithuanian Academy of Sciences, 2600 Vilnius, Akademijos St.4, Lithuania

Abstract: The smoothing constant λ is the most important characteristic of the nonparametric Parzen window classifier (PWC). The PWC tends to a one-nearest neighbour classifier as λ tends to zero and to a parametric linear Eucliden distance classifier as λ tends to infinity. An asymptotic probability of misclassification of the PWC decreases with the decrease in λ. A sensitivity of the PWC to a finiteness of the training data depends on a true-intrinsic dimensionality of the data, and it increases with the decrease in the value of λ. It is proposed to determine an optimal value of the smoothing constant from a smoothed empirical graph of the dependence of an expected probability of misclassification on the value of λ. The graph can be estimated by means of leaving-one-out or hold-out methods simultaneously for a number of values of λ chosen from the interval (0.001–1000) in a logarithmic scale.

Keywords: Parzen window elascifier, the density estimate, probability density function, kernel function, smoothing parameter, classification error, probability of misclassification

DOI: 10.3233/INF-1991-2307

Journal: Informatica, vol. 2, no. 3, pp. 434-454, 1991