SML-Bench – A benchmarking framework for structured machine learning

Issue title: Special Issue on Benchmarking Linked Data

Guest editors: Axel-Cyrille Ngonga Ngomo, Irini Fundulaki and Anastasia Krithara

Article type: Research Article

Authors: Westphal, Patrick^{a; *} | Bühmann, Lorenz^a | Bin, Simon^a | Jabeen, Hajira^b | Lehmann, Jens^{b; c}

Affiliations: [a] Computer Science Institute, University of Leipzig, Germany. E-mails: patrick.westphal@informatik.uni-leipzig.de, buehmann@informatik.uni-leipzig.de, sbin@informatik.uni-leipzig.de | [b] Computer Science Institute, University of Bonn, Germany. E-mails: jabeen@cs.uni-bonn.de, jens.lehmann@cs.uni-bonn.de | [c] Fraunhofer Institute for Intelligent Analysis and Information Systems (IAIS), Germany. E-mail: jens.lehmann@iais.fraunhofer.de

Correspondence: [*] Corresponding author. E-mail: patrick.westphal@informatik.uni-leipzig.de.

Abstract: The availability of structured data has increased significantly over the past decade and several approaches to learn from structured data have been proposed. These logic-based, inductive learning methods are often conceptually similar, which would allow a comparison among them even if they stem from different research communities. However, so far no efforts were made to define an environment for running learning tasks on a variety of tools, covering multiple knowledge representation languages. With SML-Bench, we propose a benchmarking framework to run inductive learning tools from the ILP and semantic web communities on a selection of learning problems. In this paper, we present the foundations of SML-Bench, discuss the systematic selection of benchmarking datasets and learning problems, and showcase an actual benchmark run on the currently supported tools.

Keywords: Benchmark, structured machine learning

DOI: 10.3233/SW-180308

Journal: Semantic Web, vol. 10, no. 2, pp. 231-245, 2019