Text Categorization Using Neural Networks Initialized with Decision Trees

Article type: Research Article

Authors: Remeikis, Nerijus | Skučas, Ignas | Melninkaitė, Vida

Affiliations: Faculty of Informatics, Vytautas Magnus University, Vileikos 8, 3035 Kaunas, Lithuania, e‐mail: nerijus_remeikis@fc.vdu.lt, ignas_skucas@fc.vdu.lt, vida_melninkaite@fc.vdu.lt

Abstract: Text categorization – the assignment of natural language documents to one or more predefined categories based on their semantic content – is an important component in many information organization and management tasks. Performance of neural networks learning is known to be sensitive to the initial weights and architecture. This paper discusses the use multilayer neural network initialization with decision tree classifier for improving text categorization accuracy. Decision tree from root node until a final leave is used for initialization of each single unit. Growing decision trees with increasingly larger amounts of training data will result in larger decision tree sizes. As a result, the neural networks constructed from these decision trees are often larger and more complex than necessary. Appropriate choice of certainty factor is able to produce trees that are essentially constant in size in the face of increasingly larger training sets. Experimental results support the conclusion that error based pruning can be used to produce appropriately sized trees, which are directly mapped to optimal neural network architecture with good accuracy. The experimental evaluation demonstrates this approach provides better classification accuracy with Reuters‐21578 corpus, one of the standard benchmarks for text categorization tasks. We present results comparing the accuracy of this approach with multilayer neural network initialized with traditional random method and decision tree classifiers.

Keywords: text classification, decision trees, neural networks

Journal: Informatica, vol. 15, no. 4, pp. 551-564, 2004