LCE Kotisivu

S-114.1310 Introduction to Modeling and Information Theory

Fall 2007 (3 p)



The objective of this course is to give understanding about the basic priciples of modeling, emphasizing on statistical modeling. Also information theory and its applications in statistical modeling are discussed. Practical solutions are given to model class selection, model complexity and parameter selection.

M.Sc. Janne Ojanen (lectures), M.Sc. Pasi Jylänki (exercises)
The course is held on academic year 2007-2008 on teaching period II. There are 18 hours of lectures and 6 hours of exercises. Lectures and exercises take place in Innopoli 2, B317 (the lecture room of Laboratory of Computational Engineering, see map).
Basics of probability and statistics, for example Mat-1.2600 or Mat-1.2620. Basic skills in Matlab.
Course registration
Please register for course either in WebTopi or by sending e-mail to lecturer ( If you wish to take the course in English, please contact the lecturer.
Course requirements
Examination and home assignment.

Examination schedule:

Schedule for the home assignment:

Lectures and exercises are given in Finnish. However, all the necessary course material (such as lecture slides and exercise assignments) will be available in English for English speaking participants.


List of contents:

Examination requirements

Lecture slides and exercise problems are the necessary and sufficient material for the examination. Additional course material may be helpful, but the examination will be based on slides and exercises.

Home assignment

Include your name, stundent number and email address in the assignment report. Follow the home assignment instructions on how to return the report (answers to exercises, deadlines, file formats, etc.). You can also do the assignment in pairs. In this case the pair returns a single report.

The home assignment is graded pass/fail.

Home assignment instructions PDF
Matlab files ex1_poly.m ex2_ar.m
Data sets Exercise 1 Exercise 2

Lecture slides

All lectures in one file:

Lecture slides (8 slides per page, PDF) Lecture slides (8 slides per page, PS)
Lecture slides (4 slides per page, PDF)
Lecture slides (2 slides per page, PDF)

Individual lectures (PostScript, 8 slides per page):

1.11. Lecture 1 Lecture 2
6.11. Lecture 3 Lecture 4
8.11. Lecture 5
13.11. Lecture 6 Lecture 7
15.11. Lecture 8
20.11. Lecture 9 Lecture 10
22.11. Lecture 11
27.11. Lecture 12 Lecture 13
04.12. Lecture 14 Lecture 15
11.12. Lecture 16 Lecture 17
13.12. Lecture 18


8.11. Exercise 1 (PDF) Solutions 1 (PDF)
15.11. Exercise 2 (PDF) Solutions 2 (PDF)
22.11. Exercise 3 (PDF) Solutions 3 (PDF)
29.11. Exercise 4 (PDF) Solutions 4 (PDF)
29.11. Exercise 5 (PDF) Solutions 5 (PDF)
13.12. Exercise 6 (PDF) Solutions 6 (PDF)

Additional course material

A book on information theory by David MacKay (free electronic use) can be found in . For the purposes of this course, MacKay's book contain useful material on information theory, coding and Bayesian methods.

Lecture notes from W.D. Penny's signal processing course can be found in . Chapters 1, 2.1-2.5, 3 (not 3.3), 4 ja 10.1-10.3 are useful also in this course.

An introduction into modern MDL theory can be found in the book Minimum Description Length - Theory and Applications by Peter Grünwald. The first two introductory chapters are available on his homepage by the name A Tutorial Introduction to the Minimum Description Principle.

Additional literature

More detailed material and background information; not required in the examination.

Basics of probability and statistics:

Information theory:

Statistical modeling:

Minimum Description Lenght principle:

Questions and comments:

M.Sc. Janne Ojanen
Phone: (09) 451 4837

Tästä sivusta vastaa
Sivua on viimeksi päivitetty 28.8.2007