Analytic and Stochastic Methods of Structure Parameter Estimation

Article type: Research Article

Authors: Kuznetsov, Mikhail^{; *} | Tokmakova, Aleksandra | Strijov, Vadim

Affiliations: Moscow Institute of Physics and Technology, Institutskiy lane 9, Dolgoprudny, Moscow region, 141700, Russia. E-mails: mikhail.kuznecov@phystech.edu, aleksandra-tok@yandex.ru, strijov@phystech.edu

Correspondence: [*] Corresponding author.

Abstract: The paper presents analytic and stochastic methods of structure parameters estimation for a model selection problem. Structure parameters are covariance matrices of parameters of linear and non-linear regression models. To optimize model parameters and structure parameters we maximize a model evidence, a convolution of a data likelihood with a prior distribution of model parameters. The analytic methods are based on the derivatives computation of the approximated model evidence. The stochastic methods are based on the model parameters sampling and data cross-validation. The proposed methods are tested and compared on the synthetic and real data.

Keywords: structure parameters optimization, regression model, error function, Laplace approximation, Monte-Carlo estimation, cross-validation

Journal: Informatica, vol. 27, no. 3, pp. 607-624, 2016