Bayesian Statistical Methods

Centre of Excellence in Computational Complex Systems Research

 

Gaussian processes in Bayesian modelling of complex systems

Researchers: Aki Vehtari, Jarno Vanhatalo, Jaakko Riihimäki, Jouni Hartikainen, Pasi Jylänki, Ville Pietiläinen

Gaussian processes (GP) are a flexible and attractive method for a wide variety of supervised learning problems, such as regression and classification in machine learning or spatial analysis in epidemiology. The main drawback of a Gaussian process is the fast growing need of computation time and memory requirements as the size of the training set increases.

In this project, the aim is to study approximate methods to overcome the computational limitations of a Gaussian process. At the moment the research has been in sparse approximations, which are used to speed up the otherwise heavy computations. The methods implemented this far are a Fully and partially independent conditional (FIC/PIC) and additive model using compact support covariance function and FIC (CS+FIC). Other research topic is the approximate inference using expectation probagation in models with non Gaussian likelihood.

The practical applications where the models are used are part of a New Analysis Methods for Healthcare Process Management research project. In the project, Gaussian processes are used to construct the intensity surfaces of the relative disease risk in spatial epidemiology and in the development of flexible Bayesian methods for the analysis of large scale patient data.

Software

The software written during the research is available in GPstuff.

References

  • Jarno Vanhatalo, Ville Pietiläinen and Aki Vehtari (2010). Approximate inference for disease mapping with sparse Gaussian processes. Statistics in Medicine, Accepted for publication.
  • Ville Pietiläinen (2010). Approximations for integration over the hyperparameters in Gaussian processes. M.Sc. thesis, Department of Biomedical Engineering and Computational Science, Aalto University.

  • Jarno Vanhatalo, Pasi Jylänki and Aki Vehtari (2009). Gaussian process regression with Student-t likelihood. In Bengio et al, editors, Advances in Neural Information Processing Systems 22, pp. 1910-1918, NIPS Foundation (Available online)

  • Jarno Vanhatalo and Aki Vehtari (2009). Discussion to 'Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations' by Håvard Rue, Sara Martino and Nicolas Chopin. Journal of the Royal Statistical Society, Series B (Statistical Methodology)., 71(2):383 (Available online 6 April 2009)

  • Jarno Vanhatalo and Aki Vehtari (2008). Modelling local and global phenomena with sparse Gaussian processes. Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence. (PDF)

  • Jouni Hartikainen (2008). Sparse Gaussian process models in Bayesian spatio-temporal analysis. M.Sc. Thesis, Department of Biomedical Engineering and Computational Science, Helsinki University of Technology. (PDF)

  • Jarno Vanhatalo and Aki Vehtari (2007). Sparse Log Gaussian Processes via MCMC for Spatial Epidemiology. JMLR Workshop and Conference Proceedings, 1:73-89. (Gaussian Processes in Practice) (PDF) (Slides related to the paper in PDF)

  • Jarno Vanhatalo (2006). Sparse log Gaussian process in spatial epidemiology. M.Sc. Thesis, Department of Electrical and Communications Engineering, Helsinki University of Technology. (PDF)