LCE Kotisivu

S-114.1310 Introduction to Modeling and Information Theory

Fall 2007 (3 p)

Suomeksi


NEWS:


The objective of this course is to give understanding about the basic priciples of modeling, emphasizing on statistical modeling. Also information theory and its applications in statistical modeling are discussed. Practical solutions are given to model class selection, model complexity and parameter selection.

Teachers
M.Sc. Janne Ojanen (lectures), M.Sc. Pasi Jylänki (exercises)
Schedule
The course is held on academic year 2007-2008 on teaching period II. There are 18 hours of lectures and 6 hours of exercises. Lectures and exercises take place in Innopoli 2, B317 (the lecture room of Laboratory of Computational Engineering, see map).
Prerequisites
Basics of probability and statistics, for example Mat-1.2600 or Mat-1.2620. Basic skills in Matlab.
Course registration
Please register for course either in WebTopi or by sending e-mail to lecturer (jiojanen@lce.hut.fi). If you wish to take the course in English, please contact the lecturer.
Course requirements
Examination and home assignment.

Examination schedule:

Schedule for the home assignment:

Language
Lectures and exercises are given in Finnish. However, all the necessary course material (such as lecture slides and exercise assignments) will be available in English for English speaking participants.

Contents

List of contents:

Examination requirements

Lecture slides and exercise problems are the necessary and sufficient material for the examination. Additional course material may be helpful, but the examination will be based on slides and exercises.

Home assignment

Include your name, stundent number and email address in the assignment report. Follow the home assignment instructions on how to return the report (answers to exercises, deadlines, file formats, etc.). You can also do the assignment in pairs. In this case the pair returns a single report.

The home assignment is graded pass/fail.

Home assignment instructions PDF
Matlab files ex1_poly.m ex2_ar.m
Data sets Exercise 1 Exercise 2

Lecture slides

All lectures in one file:

Lecture slides (8 slides per page, PDF) Lecture slides (8 slides per page, PS)
Lecture slides (4 slides per page, PDF)
Lecture slides (2 slides per page, PDF)

Individual lectures (PostScript, 8 slides per page):

1.11. Lecture 1 Lecture 2
6.11. Lecture 3 Lecture 4
8.11. Lecture 5
13.11. Lecture 6 Lecture 7
15.11. Lecture 8
20.11. Lecture 9 Lecture 10
22.11. Lecture 11
27.11. Lecture 12 Lecture 13
04.12. Lecture 14 Lecture 15
11.12. Lecture 16 Lecture 17
13.12. Lecture 18

Exercises

8.11. Exercise 1 (PDF) Solutions 1 (PDF)
15.11. Exercise 2 (PDF) Solutions 2 (PDF)
22.11. Exercise 3 (PDF) Solutions 3 (PDF)
29.11. Exercise 4 (PDF) Solutions 4 (PDF)
29.11. Exercise 5 (PDF) Solutions 5 (PDF)
13.12. Exercise 6 (PDF) Solutions 6 (PDF)

Additional course material

A book on information theory by David MacKay (free electronic use) can be found in http://www.inference.phy.cam.ac.uk/mackay/itila/book.html . For the purposes of this course, MacKay's book contain useful material on information theory, coding and Bayesian methods.

Lecture notes from W.D. Penny's signal processing course can be found in http://www.fil.ion.ucl.ac.uk/~wpenny/course/course.html . Chapters 1, 2.1-2.5, 3 (not 3.3), 4 ja 10.1-10.3 are useful also in this course.

An introduction into modern MDL theory can be found in the book Minimum Description Length - Theory and Applications by Peter Grünwald. The first two introductory chapters are available on his homepage http://homepages.cwi.nl/~pdg/publicationpage.html by the name A Tutorial Introduction to the Minimum Description Principle.


Additional literature

More detailed material and background information; not required in the examination.

Basics of probability and statistics:

Information theory:

Statistical modeling:

Minimum Description Lenght principle:


Questions and comments:

M.Sc. Janne Ojanen
TKK/LCE
Phone: (09) 451 4837
email: jiojanen@lce.hut.fi


Tästä sivusta vastaa jiojanen@lce.hut.fi
Sivua on viimeksi päivitetty 28.8.2007
URL: http://www.lce.hut.fi/teaching/S-114.1310/