Affiliations: School of Information Technology and Engineering, University of Ottawa, 150 Louis Pasteur, P.O. Box 450 Stn. A, Ottawa, Ontario, Canada, B3H 1W5
Note: [1] This research was supported by an NSERC grant and a grant from the University of Ottawa. We would like to thank the
anonymous reviewers for their thoughtful comments as well as the TAMALE Seminar audience at the University of Ottawa,
especially Chris Drummond, Rob Holte and Andrew McPherson who suggested many useful experiments during earlier
presentations of this work.
Abstract: In machine learning problems, differences in prior class probabilities -- or class imbalances -- have been reported to hinder the performance of some standard classifiers, such as decision trees. This paper presents a systematic study aimed at answering three different questions. First, we attempt to understand the nature of the class imbalance problem by establishing a relationship between concept complexity, size of the training set and class imbalance level. Second, we discuss several basic re-sampling or cost-modifying methods previously proposed to deal with the class imbalance problem and compare their effectiveness. The results obtained by such methods on artificial domains are linked to results in real-world domains. Finally, we investigate the assumption that the class imbalance problem does not only affect decision tree systems but also affects other classification systems such as Neural Networks and Support Vector Machines.
Keywords: concept learning, class imbalances, re-sampling, misclassification costs, C5.0, Multi-Layer Perceptrons, Support Vector Machines
