Multiclass spectral feature scaling method for dimensionality reduction

Article type: Research Article

Authors: Matsuda, Momo^* | Morikuni, Keiichi | Imakura, Akira | Ye, Xiucai | Sakurai, Tetsuya

Affiliations: University of Tsukuba, Tsukuba, Ibaraki, Japan

Correspondence: [*] Corresponding author: Momo Matsuda, Department of Computer Science, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, Japan. Tel.: +81 29 853 6574; Fax: +81 29 853 6574; E-mail: matsuda.momo.ww@alumni.tsukuba.ac.jp.

Abstract: Irregular features disrupt the desired classification. In this paper, we consider aggressively modifying scales of features in the original space according to the label information to form well-separated clusters in low-dimensional space. The proposed method exploits spectral clustering to derive scaling factors that are used to modify the features. Specifically, we reformulate the Laplacian eigenproblem of the spectral clustering as an eigenproblem of a linear matrix pencil whose eigenvector has the scaling factors. Numerical experiments show that the proposed method outperforms well-established supervised dimensionality reduction methods for toy problems with more samples than features and real-world problems with more features than samples.

Keywords: Dimensionality reduction, spectral clustering, feature scaling, machine learning

DOI: 10.3233/IDA-194942

Journal: Intelligent Data Analysis, vol. 24, no. 6, pp. 1273-1287, 2020