Combining multiple predictive models using genetic algorithms

Issue title: Combined Learning Methods and Mining Complex Data

Article type: Research Article

Authors: Janusz, Andrzej

Affiliations: Faculty of Mathematics, Informatics, and Mechanics, The University of Warsaw, Banacha 2, 02-097 Warszawa, Poland. E-mail: andrzejanusz@gmail.com

Abstract: Blending is a well-established technique, commonly used to increase performance of predictive models. Its effectiveness has been confirmed in practice as most of the latest international data-mining contest winners were using some kind of a committee of classifiers to produce their final entry. This paper presents a method of using a genetic algorithm to optimize an ensemble of multiple classification or regression models. An implementation of that method in R system, called Genetic Meta-Blender, was tested during the Australasian Data Mining 2009 Analytic Challenge. A subject of this data mining competition was the methods for combining predictive models. The described approach was awarded with the Grand Champion prize for achieving the best overall result. In this paper, the purpose of the challenge is described and details of the winning approach are given. The results of Genetic Meta-Blender are also discussed and compared to several baseline scores. Additionally, GMB is evaluated on data from a different data mining competition, namely SIAM SDM'11 Contest: Prediction of Biological Properties of Molecules from Chemical Structure.

Keywords: Classification ensemble, predictive models, genetic optimization, meta-learning

DOI: 10.3233/IDA-2012-0550

Journal: Intelligent Data Analysis, vol. 16, no. 5, pp. 763-776, 2012