A realistic evaluation of indoor positioning systems based on Wi-Fi fingerprinting: The 2015 EvAAL–ETRI competition

Article type: Research Article

Authors: Torres-Sospedra, Joaquín^{a; *; **} | Moreira, Adriano^{b; **} | Knauth, Stefan^{c; **} | Berkvens, Rafael^{d; **} | Montoliu, Raul^a | Belmonte, Oscar^a | Trilles, Sergio^a | João Nicolau, Maria^b | Meneses, Filipe^{b; e} | Costa, António^b | Koukofikis, Athanasios^c | Weyn, Maarten^d | Peremans, Herbert^f

Affiliations: [a] Institute of New Imaging Technologies, Universitat Jaume I, Castellón, Spain. E-mails: jtorres@uji.es, montoliu@uji.es, belfern@uji.es, strilles@uji.es | [b] Algoritmi Research Centre, University of Minho, Portugal. E-mails: adriano.moreira@algoritmi.uminho.pt, joao.nicolau@algoritmi.uminho.pt, antonio.costa@algoritmi.uminho.pt | [c] Faculty for Computer Science, Mathematics and Geomatics, HFT Stuttgart – Stuttgart University of Applied Sciences, Schellingstr. 24, 70174 Stuttgart, Germany. E-mails: stefan.knauth@hft-stuttgart.de, athanasios.koukofikis@hft-stuttgart.de | [d] IDLab, Faculty of Applied Engineering, University of Antwerp – imec, Groenenborgerlaan 171, 2020 Antwerpen, Belgium. E-mails: rafael.berkvens@uantwerpen.be, maarten.weyn@uantwerpen.be | [e] Centro de Computação Gráfica (CCG), Portugal. E-mail: filipe.meneses@ccg.pt | [f] ENM, Faculty of Applied Economics, University of Antwerp, Prinsstraat 13, 2000 Antwerpen, Belgium. E-mail: herbert.peremans@uantwerpen.be

Correspondence: [*] Corresponding author. E-mail: jtorres@uji.es.

Note: [**] Joaquín Torres-Sospedra, Adriano Moreira, Stefan Knauth and Rafael Berkvens contributed equally in the elaboration of this work as first authors.

Abstract: This paper presents results from comparing different Wi-Fi fingerprinting algorithms on the same private dataset. The algorithms where realized by independent teams in the frame of the off-site track of the EvAAL–ETRI Indoor Localization Competition which was part of the Sixth International Conference on Indoor Positioning and Indoor Navigation (IPIN 2015). Competitors designed and validated their algorithms against the publicly available UJIIndoorLoc database which contains a huge reference- and validation data set. All competing systems were evaluated using the mean error in positioning, with penalties, using a private test dataset. The authors believe that this is the first work in which Wi-Fi fingerprinting algorithm results delivered by several independent and competing teams are fairly compared under the same evaluation conditions. The analysis also comprises a combined approach: Results indicate that the competing systems where complementary, since an ensemble that combines three competing methods reported the overall best results.

Keywords: Wi-Fi fingerprinting, off-site competition, indoor positioning

DOI: 10.3233/AIS-170421

Journal: Journal of Ambient Intelligence and Smart Environments, vol. 9, no. 2, pp. 263-279, 2017