LCE Homepage

S-114.300 Topical Lectures on Information Theory and Modeling

Teachers:

Volume:

Dates and Place:

Contents:

  1. SHANNON'S INFORMATION (COMPLEXITY)
    - Basics of Coding
    - Kraft Inequality
    - Shannon's Theorem
  2. BASICS PROPERTIES OF ENTROPY, RELATIVE ENTROPY AND MUTUAL INFORMATION
    - Channel Capacity
    - Equipartition Property
  3. UNIVERSAL CODING
    - Lempel-Ziv Codes
    - Algorithm Context
    - Arithmetic Codes
  4. KOLMOGOROV COMPLEXITY
    - Universal Algorithmic Models
    - Kolmogorov Sufficient Statistics
  5. STOCHASTIC COMPLEXITY AND INFORMATION IN DATA
    - Universal Probability Models
    - Maxmin Problems
    - Prediction and Coding Bounds
    - MDL principle
  6. APPLICATIONS
    - Linear regression
    - Denoising

Literature:

  1. Jorma Rissanen's lecture note: ``Lectures on Information Theory and Modeling''.
    You can download the lecture note here:
    ( lectures.ps ) ( lectures.pdf )
  2. Thomas M. Cover and Joy A. Thomas: ``Elements of Information Theory'', John Wiley & Sons, 1991.

Exercise work:

You can download the exercise work description here:

Other information:

For registration and any other information contact:

Tommi Nykopp, M.Sc.
Laboratory of Computational Engineering
Helsinki University of Technology
Tel: 09-451 4843
Fax: 09-451 4830
Email: tnykopp@cc.hut.fi


This page is maintained by tnykopp@cc.hut.fi
This page has been updated 13.05.2002
URL: http://www.lce.hut.fi/teaching/S-114.300/