Modelling of Learning and Perception

Centre of Excellence in Computational Complex Systems Research


Bayesian Object Matching

Researchers: Toni Tamminen and Jouko Lampinen

The goal of the project is to develop a system that can locate and recognize objects in a natural scene. In our approach we study model based methods using full Bayesian inference. The objects in a scene are defined by prior models that are learned from example images.

We have developed a distortion tolerant feature matching method based on probability distributions of Gabor filter responses. An object is defined as a set of locations, with associated Gabor-features, and a prior model that defines the variations of the feature locations. The eigenshape model corresponds to determining the covariance matrix for the feature locations, which is learned in bootstrap fashion by matchihg a large number of images by simpler prior models.

For exploring the posterior distribution, we have constructed efficient MCMC samplers for drawing samples from the posterior distributions of the feature locations, mainly using Gibbs and Metropolis sampling. We have also developed a sequential Monte Carlo approach which handles multimodal posterior distributions better than MCMC samplers. This is especially important when some of the object features are occluded, as is often the case in real matching situations. Currently we are extending the matching model to multiple resolutions, which would allow the matching of objects of greatly varying sizes.

Figure 1 illustrates the object shape model by showing the first few eigenshapes and Figure 2 shows an example of the sequential matching process. Figure 3 shows some matching results when the target objects are occluded.

Figure 1

Figure 1. Leading eigenshapes of faces, learnt from a set of training images. The face on the left has been morphed according to the eigen shapes, into positive direction (upper row) and negative direction (lower row). It can be seen that components 2 and 3 are related to rotations of the head, while components 1, 4, and 5 are shape-related.

Figure 2

Figure 2. Sequential feature matching. The black circles mark the drawn locations of the current feature, while the green circles are the previously drawn features. The shape (yellow lines) represent the mean of the shape prior.

Figure 3

Figure 3. Matching results for occluded objects. The black lines show the sample means. The matching has failed only the first image of the second row. This is because the likelihoods are very low also for the visible features - for example, the eye features are distorted by the eyeglasses, and also the chin line is very faint. In the other 5 images, the system has been able to locate the objects despite heacy occlusion.


  • Tamminen, T. and Lampinen, J. (2004). A Bayesian occlusion model for sequential object matching. In A. Hoppe, S. Barman, and T. Ellis, editors, Proceedings of the British Machine Vision Conference 2004. The paper was awarded the Model-Based Vision Prize BMVC 2004.

  • Tamminen, T. and Lampinen, J. (2003). Bayesian object matching with hierarchical priors and Markov chain Monte Carlo. In J. M. Bernardo et al., editors, Bayesian Statistics 7. Oxford University Press.

  • Tamminen, T. and Lampinen, J. (2003). Learning an object model for feature matching in clutter. In Bigun, J. and Gustavsson, T., editors, Proceedings of the 13th Scandinavian Conference on Image Analysis, SCIA 2003. Springer.