Heat Map Based Feature Ranker: In depth comparison with popular methods

Article type: Research Article

Authors: Huertas, Carlos^{a; *} | Juarez-Ramirez, Reyes^a | Raymond, Christian^b

Affiliations: [a] Universidad Autonoma de Baja California, Tijuana, Mexico | [b] Institut de Recherche en Informatique et Systemes Aleatoires, Rennes Cedex, France

Correspondence: [*] Corresponding author: Carlos Huertas, Universidad Autonoma de Baja California, Tijuana, Mexico. E-mail: chuertas@uabc.edu.mx.

Abstract: The new era of technology allows us to gather more data than ever before, complex data emerge and a lot of noise can be found among high dimensional datasets. In order to discard useless features and help build more generalized models, feature selection seeks a reduced subset of features that improve the performance of the learning algorithm. The evaluation of features and their interactions are an expensive process, hence the need for heuristics. In this work, we present Heat Map Based Feature Ranker, an algorithm to estimate feature importance purely based on its interaction with other variables. A compression mechanism reduces evaluation space up to 66% without compromising efficacy. Our experiments show that our proposal is very competitive against popular algorithms, producing stable results across different types of data. We also show how noise reduction through feature selection aids data visualization using emergent self-organizing maps.

Keywords: Feature selection, high-dimensional data, self-organizing maps

DOI: 10.3233/IDA-173481

Journal: Intelligent Data Analysis, vol. 22, no. 5, pp. 1009-1037, 2018