Affiliations: [a] The School of Computer Science, The Academic College of Tel Aviv-Yaffo, Israel | [b] Department of Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel | [c] The Efi Arazi School of Computer Science, Interdisciplinary Center Herzliya, Herzliya, Israel
Correspondence:
[*]
Corresponding author: Alon Schclar, The School of Computer Science, The Academic College of Tel Aviv-Yaffo, Israel P.O.B. 8401, Tel Aviv 61083, Israel. Tel.: +972 3 6803408; Fax: +972 3 6803342; E-mail: alonschc@mta.ac.il.
Abstract: We present a novel approach for the construction of ensemble classifiers based on dimensionality reduction. The ensemble members are trained based on dimension-reduced versions of the training set. In order to classify a test sample, it is first embedded into the dimension reduced space of each individual classifier by using an out-of-sample extension algorithm. Each classifier is then applied to the embedded sample and the classification is obtained via a voting scheme. We demonstrate the proposed approach using the Random Projections, the Diffusion Maps and the Random Subspaces dimensionality reduction algorithms. We also present a multi-strategy ensemble which combines AdaBoost and Diffusion Maps. A comparison is made with the Bagging, AdaBoost, Rotation Forest ensemble classifiers and also with the base classifier. Our experiments used seventeen benchmark datasets from the UCI repository. The results obtained by the proposed algorithms were superior in many cases to other algorithms.
Keywords: Ensembles of classifiers, dimensionality reduction, out-of-sample extension, Random Projections, Diffusion Maps, Nyström extension
