DIKEA: Exploiting Wikipedia for keyphrase extraction

Article type: Research Article

Authors: Wang, David X. | Gao, Xiaoying^{; *} | Andreae, Peter

Affiliations: Victoria University of Wellington, New Zealand. E-mails: xiao.wang.saturnv@gmail.com, xgao@ecs.vuw.ac.nz, pondy@ecs.vuw.ac.nz

Correspondence: [*] Corresponding author. E-mail: xgao@ecs.vuw.ac.nz.

Abstract: Automatic keyphrase extraction is the challenging task of assigning keyphrases to documents to capture the main topics. It assists many research areas in the field of text mining – indexing, clustering, and summarisation. A landmark research KEA (Keyphrase Extraction Algorithm) formulated the problem as a supervised machine learning problem and successfully applied a Naïve Bayes model to it. KEA showed great promise but its performance is not satisfactory. Its state-of-art extension KEA++ significantly improved its performance but relies on a domain specific vocabulary which is often not available or incomplete for other domains. We present a novel domain-independent system (DIKEA) which makes three main contributions to this field of research: utilising the largest online knowledge source available, Wikipedia, for keyphrase candidate selection; adding new features including a Wikipedia-based feature, link probability; and further boosting performance by using a multilayer perceptron network. Our experiments showed that DIKEA outperformed KEA++ while keeping the overall solution domain-independent. DIKEA was also tested on a benchmark dataset provided by a workshop on Semantic Evaluation (SemEval-2010), allowing comparisons with the 19 other related systems which participated. Our experiments show that DIKEA ranks first when considering only the top 5 keyphrases extracted from each document, and ranks second overall.

Keywords: Keyword extraction, Wikipedia-based, domain independent

DOI: 10.3233/WEB-150318

Journal: Web Intelligence, vol. 13, no. 3, pp. 153-165, 2015