Affiliations: Department of Computer Science, The University of Waikato, New Zealand
Note: [1] E-mail: Kaiming@cs.waikato.ac.nz.
Abstract: In this article, we first explore an intrinsic problem that exists in the models induced by learning algorithms. Regardless of the selected algorithm, search methodology and hypothesis representation by which the model is induced, one would expect the model to make better predictions in some regions of the description space than others. We present the fact that an induced model will have some regions of relatively poor performance: the problem of locally low predictive accuracy. Holte, Arker, Porter [21] addressed this intrinsic problem in learning systems that describe the induced model as a disjunction of conjunctions of conditions. In this article, we investigate the characterisation of the problem in instance-based and Naive Bayesian classifiers. Having characterised the problem of locally low predictive accuracy, we propose to counter the problem in these two types of learning algorithms, using a composite learner framework. The strategy is to select an estimated better performing model to do the final prediction during classification. Empirical results from fifteen real-world domains show that the strategy is capable of partially overcoming the problem of locally low predictive accuracy, and at the same time improving the overall performance of its constituent algorithms in most of the domains studied. The composite learner is also found to outperform four methods of stacked generalisation, and also a model selection method based on cross-validation, in most of the experimental domains studied.
