Searching for just a few words should be enough to get started. If you need to make more complex queries, use the tips below to guide you.
Issue title: Intelligent and Fuzzy Systems applied to Language & Knowledge Engineering
Guest editors: David Pinto, Vivek Kumar Singh, Aline Villavicencio, Philipp Mayr-Schlegel and Efstathios Stamatatos
Article type: Research Article
Authors: Jimenez, Sergioa; * | Cucerzan, Silviu-Petrub | Gonzalez, Fabio A.c | Gelbukh, Alexanderd | Dueñas, Georgea
Affiliations: [a] Instituto Caro y Cuervo, Calle 10 #4-69, Bogotá, D.C., Colombia | [b] Microsoft Research, One Microsoft Way, Redmond, WA, US | [c] Universidad Nacional de Colombia, Ciudad Universitaria, Bogotá, D.C., Colombia | [d] CIC, Instituto Politécnico Nacional, Av. Juan de Dios Bátiz, 07738, Mexico City, Mexico
Correspondence: [*] Corresponding author. Sergio Jimenez, Instituto Caro y Cuervo, Calle 10 #4-69, Bogotá, D.C., Colombia. E-mail: sergio.jimenez@caroycuervo.gov.co.
Abstract: In this paper, the use of collection term frequencies (i.e. the total number of occurrences of a term in a document collection) in the BM25 retrieval model is investigated by modifying its term frequency (TF) and inverse document frequency (IDF) components. Using selected examples extracted from TREC collections, it was observed that the informative nature, for retrieval purposes, of terms, either with the same TF (in a document) or IDF (in a collection) may be better revealed with the use of collection term frequencies (CTF). From three new heuristics based on those observations and deviations from a random Poisson model, collection term frequencies were integrated to TF and IDF factors. The novel formulations were tested by employing the TREC-1 to TREC-8 collections in the ad hoc task, for which BM25 was first developed and tested. Consistent and significant improvements were observed in mean average precision (MAP) reaching up to 17.67% for the TREC-8 dataset, and 7.16% averaged over all tested collections. These results were considerably better in comparison to other approaches surveyed aiming to improve BM25, proving in this way the effectiveness of the proposed heuristics and formulae. The proposed approach requires only additional offline pre-computations and does not entail extra computational complexity for retrieval while keeping the original spirit and parameter robustness of BM25.
Keywords: BM25, tf-idf, collection term frequency, information retrieval heuristics, TREC collections, deviation from randomness
DOI: 10.3233/JIFS-169475
Journal: Journal of Intelligent & Fuzzy Systems, vol. 34, no. 5, pp. 2887-2899, 2018
IOS Press, Inc.
6751 Tepper Drive
Clifton, VA 20124
USA
Tel: +1 703 830 6300
Fax: +1 703 830 2300
sales@iospress.com
For editorial issues, like the status of your submitted paper or proposals, write to editorial@iospress.nl
IOS Press
Nieuwe Hemweg 6B
1013 BG Amsterdam
The Netherlands
Tel: +31 20 688 3355
Fax: +31 20 687 0091
info@iospress.nl
For editorial issues, permissions, book requests, submissions and proceedings, contact the Amsterdam office info@iospress.nl
Inspirees International (China Office)
Ciyunsi Beili 207(CapitaLand), Bld 1, 7-901
100025, Beijing
China
Free service line: 400 661 8717
Fax: +86 10 8446 7947
china@iospress.cn
For editorial issues, like the status of your submitted paper or proposals, write to editorial@iospress.nl
如果您在出版方面需要帮助或有任何建, 件至: editorial@iospress.nl