RODI: Benchmarking relational-to-ontology mapping generation quality

Issue title: Quality management of Semantic Web assets (data, services and systems)

Guest editors: Amrapali Zaveri, Dimitris Kontokostas, Sebastian Hellmann and Jürgen Umbrich

Article type: Research Article

Authors: Pinkel, Christoph^{a; *; **} | Binnig, Carsten^b | Jiménez-Ruiz, Ernesto^c | Kharlamov, Evgeny^c | May, Wolfgang^d | Nikolov, Andriy^a | Sasa Bastinos, Ana^a | Skjæveland, Martin G.^e | Solimando, Alessandro^f | Taheriyan, Mohsen^g | Heupel, Christian^a | Horrocks, Ian^c

Affiliations: [a] Fluid Operations AG, Walldorf, Germany | [b] Brown University, Providence, RI, USA | [c] University of Oxford, United Kingdom | [d] Göttingen University, Germany | [e] University of Oslo, Norway | [f] Inria Saclay & Université Paris-Sud, Orsay, France | [g] University of Southern California, Los Angeles, CA, USA

Correspondence: [*] Corresponding author. E-mail: contact@fluidops.com.

Note: [**] This paper is a significantly extended version of the conference paper: “RODI: A Benchmark for Automatic Mapping Generation in Relational-to-Ontology Data Integration” [39].

Abstract: Accessing and utilizing enterprise or Web data that is scattered across multiple data sources is an important task for both applications and users. Ontology-based data integration, where an ontology mediates between the raw data and its consumers, is a promising approach to facilitate such scenarios. This approach crucially relies on useful mappings to relate the ontology and the data, the latter being typically stored in relational databases. A number of systems to support the construction of such mappings have recently been developed. A generic and effective benchmark for reliable and comparable evaluation of the practical utility of such systems would make an important contribution to the development of ontology-based data integration systems and their application in practice. We have proposed such a benchmark, called RODI. In this paper, we present a new version of RODI, which significantly extends our previous benchmark, and we evaluate various systems with it. RODI includes test scenarios from the domains of scientific conferences, geographical data, and oil and gas exploration. Scenarios are constituted of databases, ontologies, and queries to test expected results. Systems that compute relational-to-ontology mappings can be evaluated using RODI by checking how well they can handle various features of relational schemas and ontologies, and how well the computed mappings work for query answering. Using RODI, we conducted a comprehensive evaluation of seven systems.

Keywords: Mappings, relational databases, RDB2RDF, R2RML, benchmarking, bootstrapping

DOI: 10.3233/SW-170268

Journal: Semantic Web, vol. 9, no. 1, pp. 25-52, 2018