Iterative Bayes

Article type: Research Article

Authors: Gama, João

Affiliations: LIACC, FEP -- University of Porto, Rua Campo Alegre, 823, 4150 Porto, Portugal. Tel.: +351 226078830; Fax: +351 226003654; E-mail: jgama@ncc.up.pt; URL: http://www.ncc.up.pt/~jgama

Abstract: Naive Bayes is a well known and studied algorithm both in statistics and machine learning. Bayesian learning algorithms represent each concept with a single probabilistic summary. In this paper we present an iterative approach to naive Bayes. The iterative Bayes begins with the distribution tables built by the naive Bayes. Those tables are iteratively updated in order to improve the probability class distribution associated with each training example. Experimental evaluation of Iterative Bayes on 27 benchmark datasets shows consistent gains in accuracy. Moreover, the update schema can take costs into account turning the algorithm cost sensitive. Unlike stratification, it is applicable to any number of classes and to arbitrary cost matrices. An interesting side effect of our algorithm is that it shows to be robust to attribute dependencies.

DOI: 10.3233/IDA-2000-4603

Journal: Intelligent Data Analysis, vol. 4, no. 6, pp. 475-488, 2000