## S-114.300 Topical Lectures on Information Theory and Modeling | |

- Prof. Jorma Rissanen, Doc. Helsinki University of Technology
- Dr.tech. Jukka Heikkonen, Lab. of Computational Engineering, HUT
- M.Sc. Tommi Nykopp, Lab. of Computational Engineering, HUT

- 2 credits, L

- Lectures and exercises:

9.4.2002 and 16.4.2002 at 9-12 room G (main building)

10.4.2002 and 17.4.2002 at 9-12 room N (main building)

11.4.2002 and 19.4.2002 at 9-12 room G (main building)

- SHANNON'S INFORMATION (COMPLEXITY)

- Basics of Coding

- Kraft Inequality

- Shannon's Theorem

- BASICS PROPERTIES OF ENTROPY, RELATIVE ENTROPY AND MUTUAL INFORMATION

- Channel Capacity

- Equipartition Property

- UNIVERSAL CODING

- Lempel-Ziv Codes

- Algorithm Context

- Arithmetic Codes

- KOLMOGOROV COMPLEXITY

- Universal Algorithmic Models

- Kolmogorov Sufficient Statistics

- STOCHASTIC COMPLEXITY AND INFORMATION IN DATA

- Universal Probability Models

- Maxmin Problems

- Prediction and Coding Bounds

- MDL principle

- APPLICATIONS

- Linear regression

- Denoising

- Jorma Rissanen's lecture note: ``Lectures on Information Theory and Modeling''.

You can download the lecture note here:

( lectures.ps ) ( lectures.pdf )

- Thomas M. Cover and Joy A. Thomas: ``Elements of Information Theory'', John Wiley & Sons, 1991.

You can download the exercise work description here:

- Excercise work instructions: ( hwork.ps )
- Excercise data: ( EEGsignal.mat )

- Requirements: Attendance in the lectures and exercises. The course also includes an exercise work related to the subject.
- Language: The course will be given in English.
- Registration: To the email address given below by 8.4.2002, or during the first lecture.

Tommi Nykopp, M.Sc.

Laboratory of Computational Engineering

Helsinki University of Technology

Tel: 09-451 4843

Fax: 09-451 4830

Email: tnykopp@cc.hut.fi

This page is maintained by tnykopp@cc.hut.fi

This page has been updated 13.05.2002

URL: http://www.lce.hut.fi/teaching/S-114.300/