Affiliations: [a] Department of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK. E-mail: huma@cs.rhul.ac.uk | [b] Global Analytics Group, Canadian Imperial Bank of Commerce, 161 Bay St., BCE-11, Toronto ON, Canada M5J 2S8. E-mail: Grigoris.Karakoulas@cibc.ca | [c] Department of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK. E-mail: john@cs.rhul.ac.uk
Abstract: This paper introduces a strategy for training ensemble classifiers by analysing boosting within margin theory. We present a bound on the generalisation error of ensembled classifiers in terms of the 2-norm of the margin slack vector. We develop an effective, adaptive and robust boosting algorithm, DMBoost, by optimising this bound. The soft margin based quadratic loss function is insensitive to points having a large margin. The algorithm improves the generalisation performance of a system by ignoring the examples having small or negative margin. We evaluate the efficacy of the proposed method by applying it to a text categorization task. Experimental results show that DMBoost performs significantly better than AdaBoost, hence validating the effectiveness of the method. Furthermore, experimental results on UCI data sets demonstrate that DMBoost generally outperforms AdaBoost.
Keywords: boosting, margin theory, soft margin, quadratic loss, text categorization
