PointMap: A Real-Time Memory-Based Learning System with On-line and Post-Training Pruning

Article type: Research Article

Authors: Kopčo, Norbert | Carpenter, Gail A.

Affiliations: Department of Cognitive and Neural Systems, Boston University, Boston, Massachusetts USA. kopco@cns.bu.edu, gail@cns.bu.edu

Abstract: A memory-based learning system called PointMap is a simple and computationally efficient extension of Condensed Nearest Neighbor that allows the user to limit the number of exemplars stored during incremental learning. PointMap evaluates the information value of coding nodes during training, and uses this index to prune uninformative nodes either on-line or after training. These pruning methods allow the user to control both a priori code size and sensitivity to detail in the training data, as well as to determine the code size necessary for accurate performance on a given data set. Coding and pruning computations are local in space, with only the nearest coded neighbor available for comparison with the input; and in time, with only the current input available during coding. Pruning helps solve common problems of traditional memory-based learning systems: large memory requirements, their accompanying slow on-line computations, and sensitivity to noise. PointMap copes with the curse of dimensionality by considering multiple nearest neighbors during testing without increasing the complexity of the training process or the stored code. The performance of PointMap is compared to that of a group of sixteen nearest-neighbor systems on benchmark problems.

Keywords: memory-based learning, nearest neighbor, on-line pruning, post-training pruning, incremental learning

DOI: 10.3233/HIS-2004-11-208

Journal: International Journal of Hybrid Intelligent Systems, vol. 1, no. 1-2, pp. 57-71, 2004