Maximum-entropy estimated distribution model for classification problems

Article type: Research Article

Authors: Tan, Ling | Taniar, David^{; *} | Smith, Kate A.

Affiliations: School of Business Systems, Monash University, Clayton, Vic 3800, Australia

Correspondence: [*] Corresponding author. Tel.: +61 3 9905 9693; Fax: +61 3 9905 5159; E-mail: david.taniar@infotech.monash.edu.au

Abstract: Classification is a fundamental problem in machine learning and data mining. This paper applies a stochastic optimization model to classification problems. The proposed maximum entropy estimated distribution model uses a probabilistic distribution to represent solution space, and a sampling technique to explore search space. This paper demonstrates the application of the proposed maximum entropy estimated distribution model to improve linear discriminant function and rule induction methods. In addition, this paper compares the proposed classification model with decision trees. It shows that the proposed model is preferable to decision tree C4.5 in the following cases: i) when prior distribution of classification is available; ii) when no assumption is made about underlying classification structure; and iii) when a classification problem is multimodal in nature.

Keywords: Estimated distribution algorithms, hybrid evolutionary algorithms, multimodal classification, data mining

DOI: 10.3233/HIS-2006-3101

Journal: International Journal of Hybrid Intelligent Systems, vol. 3, no. 1, pp. 1-10, 2006