Rectifying the representation learned by Non-negative Matrix Factorization

Article type: Research Article

Authors: Yoshida, Tetsuya

Affiliations: Graduate School of Information Science and Technology, Hokkaido University, N-14, W-9, Sapporo, Hokkaido 060-0814, Japan. Tel.: +81 11 706 7253; Fax: +81 11 706 7808; E-mail: yoshida@meme.hokudai.ac.jp

Abstract: This paper proposes a novel method to the problem of non-orthogonality of features obtained using the Non-negative Matrix Factorization (NMF) method. For any given non-negative data matrix, the NMF method provides a learned local representation by approximating the data matrix as a product of two non-negative matrices. However, the non-orthogonality of the features hinders the effective use of the learned representation from the NMF. To overcome this problem, we propose the following steps: calculate the metric in the feature space adapted to the features, apply the Cholesky decomposition to the metric and identify the upper triangular matrix, and use the upper triangular matrix as a linear mapping for the learned representation from the NMF. The proposed method is applied to current NMF-based clustering algorithms and evaluated over real-world datasets. The results indicate that the proposed method improves the performance of those algorithms, and that the method is very robust, even in the presence of a large number of features.

Keywords: Non-negative matrix factorization, representation learning, Cholesky decomposition

DOI: 10.3233/KES-130278

Journal: International Journal of Knowledge-based and Intelligent Engineering Systems, vol. 17, no. 4, pp. 279-290, 2013