Affiliations: ORT Braude College of Engineering, 21982 Karmiel, Israel, e-mail: zbarzily@braude.ac.il, vlvolkov@braude.ac.il | Institute of Applied Mathematics, Middle East Technical University, 06531 Ankara, Turkey, e-mail: bozturk@metu.edu.tr | Faculty of Economics, Business and Law, University Siegen, Hölderlinstrasse 3, 57076 Germany, e-mail: gweber@metu.edu.tr
Abstract: In this paper, a method for the study of cluster stability is purposed. We draw pairs of samples from the data, according to two sampling distributions. The first distribution corresponds to the high density zones of data-elements distribution. Thus it is associated with the clusters cores. The second one, associated with the cluster margins, is related to the low density zones. The samples are clustered and the two obtained partitions are compared. The partitions are considered to be consistent if the obtained clusters are similar. The resemblance is measured by the total number of edges, in the clusters minimal spanning trees, connecting points from different samples. We use the Friedman and Rafsky two sample test statistic. Under the homogeneity hypothesis, this statistic is normally distributed. Thus, it can be expected that the true number of clusters corresponds to the statistic empirical distribution which is closest to normal. Numerical experiments demonstrate the ability of the approach to detect the true number of clusters.
Keywords: clustering, cluster validation, minimal spanning tree, two sample test
Journal: Informatica, vol. 20, no. 2, pp. 187-202, 2009
