Diversity, accuracy and efficiency in ensemble learning: An unexpected result

Article type: Research Article

Authors: Johnson, Joseph^* | Giraud-Carrier, Christophe

Affiliations: Department of Computer Science, Brigham Young University, Provo, UT, USA

Correspondence: [*] Corresponding author: Joseph Johnson, Department of Computer Science, Brigham Young University, Provo, UT 84602, USA. Tel.: +1 801 422 3027; Fax: +1 801 422 0169; E-mail: josephjohnson11@gmail.com.

Abstract: Empirical evidence suggests that ensembles with adequate levels of pairwise diversity among a set of accurate member algorithms can significantly outperform any of the individual algorithms. As a result, several diversity measures have been developed for use in optimizing ensembles. We show, however, that there is natural tension between the pairwise diversity of ensemble members and their individual accuracy. While efficient ensembles can be built with stronger forms of diversity, they also suffer in overall accuracy. On the other hand, ensembles built with weaker forms of diversity can be very accurate, but tend to be significantly more computationally expensive. We discuss these findings in light of the notion of diversity space.

Keywords: Diversity, ensemble learning, metalearning

DOI: 10.3233/IDA-183934

Journal: Intelligent Data Analysis, vol. 23, no. 2, pp. 297-311, 2019