A comprehensive review of recursive Naïve Bayes Classifiers

Article type: Research Article

Authors: Rooney, Niall | Patterson, David | Galushka, Mykola

Affiliations: Nikel, University of Ulster, Shore rd, Newtonabbey, BT37 OQB, UK. Tel.: +44 2890366503; E-mail: NF.ROONEY@ulster.ac.uk, WD.PATTERSON@ulster.ac.uk, MG.GALUSHKA@ulster.ac.uk

Abstract: In this paper we provide a comprehensive empirical review of a variant of the Recursive Naïve Baye Classifier (RNBC*) in comparison to simple Naïve Bayes and C4.5. We show that in terms of a zero one loss cost function for classification accuracy, RNBC* outperformed Naïve Bayes and was comparable to C4.5, for the range of data-sets tested. As the Naïve Bayes classifier has been shown to be a robust classifier in many domains, this is a significant result. We estimate the bias variance decomposition of RNBC* and show that the bias-variance profile of RNBC* is more similar to that of decision trees than Naïve Bayes. We demonstrate how variance reducing ensemble techniques such as Bagging and Boosting can be effective in increasing the classification accuracy of RNBC*.

Keywords: classification, Naïve bayes, recursion, tree learning methods

DOI: 10.3233/IDA-2004-8607

Journal: Intelligent Data Analysis, vol. 8, no. 6, pp. 615-628, 2004