A new goodness-of-fit statistical test

Article type: Research Article

Authors: Apolloni, Bruno | Bassis, Simone

Affiliations: Università degli Studi di Milano, Dipartimento di Scienze dell'Informazione, via Comelico 39/41, 20135 Milano, Italy. E-mail: apolloni@dsi.unimi.it, bassis@dsi.unimi.it

Abstract: We introduce a new concept of nonparametric test for statistically deciding if a model fits a sample of data well. The employed statistic is the empirical cumulative distribution (e.c.d.f.) of the measure of the blocks determined by the ordered sample. For any distribution law underlying the data this statistic is distributed around a Beta cumulative distribution law (c.d.f.) so that the shift between the two curves is the statistic at the basis of the test. Its distribution is computed through a new bootstrap procedure from a population of free parameters of the model that are compatible with the sampled data according to the model. Closing the loop, we may expect that if the model fits the data well the Beta c.d.f. constitutes a template for the block e.c.d.f.s that are compatible with the observed data. In the paper we show how to appreciate the template functionality in the case of a good fit and also how to discriminate bad models. We show the test's potential in comparison to conventional tests, both in case studies and in a well-known benchmark for the semiparametric logistic model used widely in database analysis.

Keywords: Nonparametric statistical test, goodness-of-fit test, beta distribution, tolerance regions, algorithmic inference, decision theory

DOI: 10.3233/IDT-2007-1404

Journal: Intelligent Decision Technologies, vol. 1, no. 4, pp. 205-218, 2007