Mining a large database with a parallel database server

Article type: Research Article

Authors: de Sousa, Mauro Sérgio R. | Mattoso, Marta | Ebecken, Nelson F.F.^{; *}

Affiliations: Civil Engineering Department, Centro de Technologia, COPPE/UFRJ Cidade Universitaria, Caixa Postal 68506, 21945-970 Rio de Janeiro, RJ, Brazil

Correspondence: [*] Corresponding author. Fax: +55-21-280-9545. E-mail addresses: mauros@cos.ufrj.br (M.S.R. de Sousa), marta@cos.ufrj.br (M. Mattoso), nelson@ntt.ufrj.br (N.F.F. Ebecken).

Abstract: Data mining is a data-intensive computation activity. Parallel processing has often been used in data mining algorithms. However, when data do not fit in memory, some solutions do not apply and a database system may be required rather than flat files. Most of the implementations use the database system loosely coupled with the data mining techniques. Hence, the database system only issues queries to be processed on the client machine. In this work, we address the data consuming activities through parallel processing on a database server providing a tight integration with data mining techniques. Experimental results showing the potential benefits of this integration were obtained. Despite the difficulties in processing a complex application, we extracted rules and obtained high performance on all the data-intensive activities such as the construction of the decision tree, pruning and rule extraction.

Keywords: Data mining, Parallel processing, Parallel database server

DOI: 10.3233/IDA-1999-3603

Journal: Intelligent Data Analysis, vol. 3, no. 6, pp. 437-451, 1999