LCE Homepage

S-114.600 Introduction to Bayesian Modeling

 

Model checking

Estimating the speed of light. Simon Newcomb measured the speed of light in 1882 by inspecting the time it takes for a ray of light to travel 7442 meters. The measurements are given as deviations from 24800 nanoseconds.

Fit a Gaussian distribution to the data (unknown mean and variance) and perform some model checking. Since there are many measurements (n=66) you can use the maximum likelihood solution, i.e. compute the mean and standard deviation of the data.

The data is in the file check.dat. You can produce 1000 replicate data points from the posterior predictive distribution with e.g.

d=load('check.dat','-ascii');
R = normrnd(mean(d),std(d),length(d),1000)
R is a length(d)*1000 matrix whose columns contain the replicated data (ie. each column has 66 samples from the posterior predictive distribution). You can examine the properties of the replicated data with e.g.
 q=min(R);
Here q is a 1x1000 vector containing the minima of the replicated data points.

The posterior predictive p-value can be computed with

 mean(q >min(d))
q>min(d) returns a boolean vector which shows whether the replicated test value was greater than the value computed from the data, and mean computes how many of the test values was larger divided by the number of replications.

You can also plot an image

hist(q)
line([min(d) min(d)],get(gca,'YLim'))

In model checking we compare the data to posterior predictive distributions. Compare how some characteristics of the data and a replicated data set of the same size drawn from the posterior predictive distribution differ. Examine at least the (you can invent other ways, also)

Plot an image with the histogram of several simulations from the posterior predictive distribution and the corresponding value computed from the data.

Consider the following questions:

Note that this is a very simple data set, and in real problems the discrepancies between the model and the data can be much harder to spot. Still, these tricks are usually worth a shot.


This page is maintained by Toni.Tamminen@hut.fi
This page has been updated 10. 11. 2003
URL: http://www.lce.hut.fi/teaching/S-114.600/ex/check_en.html