
S114.4220 Research Seminar on Computational Science 


The topic of the course in fall 2007 is
Stochastic and Adaptive Control of Uncertain Systems (5 p) L V
Topic of the seminar is automatic control of uncertain dynamic
systems, where a control signal is generated based on observations
made from the system and the control signal is used for actively
changing the behaviour of the system in order to reach a certain
predefined goal. Examples of such control tasks are guidance of a
spacecraft, control of a chemical process and steering of a mobile
robot.
In the seminar, the purpose is:
 to present a review of optimal control, optimal estimation and
stochastic control. In these classical approaches, the
uncertainties are modeled with random processes and controls are
selected to optimize certain model criteria either using variational
calculus, or HamiltonJacobiBellman equations and Bayesian decision
theory.
 to present review of adaptive and learning control
approaches. In these methods, either the value functions (costtogo
functions) are directly optimized based on a model or some other means
are used for adapting the model to the observed environment.
 to find the connection between the above two approaches and the
connection to the classical pole placement based control design
methods.
 Teachers
 Dr.Tech. Simo Särkkä,
Ph.D. Tuomas Lukka
 Prerequisites

Basics of probability theory, differential equations, matrix algebra and
Bayesian inference. Basic knowledge or ability to learn to use
Matlab is needed for completing the exercise.
 Place and time

The seminars take place on Mondays at 12:15 in Innopoli 2, B317
(the seminar room of Laboratory of computational engineering, see
map).
The first lecture is 17.09.2007.
 Course Registration
 Please register for the course by sending email to
ssarkka@lce.hut.fi
Remember to mention the student id. If you
have an suggestion for the topic of your presentation, you may
mention it in the email also.
 Language

English if at least one English speaking participant.
Contents
Course requirements:
 Active participation in the seminars
 Seminar presentation
 Homework exercises (pen & paper)
 Exercise work (computer simulation)
Books
 Control engineering and optimal control:

 [1] D. E. Kirk (1970), Optimal Control Theory. Prentice Hall
(Dover 2004).
 [2] B. D. O. Anderson and J. B. Moore (1989),
Optimal Control: Linear Quadratic Methods. PrenticeHall (Dover 2007).
(Scanned PDF).
 [3] R. C. Dorf and R. H. Bishop (2004), Modern Control Systems,
Prentice Hall.
 [4] T. Glad and L. Ljung (2000), Control Theory: Multivariable
and Nonlinear Methods, Taylor & Francis.
 Stochastic optimal control and estimation:

 [5] R. Stengel (1994), Optimal Control and Estimation,
Dover Publications, New York.
 [6] B. D. O. Anderson and J. B. Moore (1979),
Optimal Filtering. PrenticeHall (Dover 2005).
(Scanned PDF).
 [7] P. Maybeck (1982). Stochastic Models, Estimation and
Control, Volume 3. Academic Press.
 [8] M. Aoki (1967), Optimization of Stochastic Systems. Academic
Press, New York.
 Approximate dynamic programming:

 [9] D. P. Bertsekas (2001), Dynamic Programming and Optimal Control,
Volume 1, Athena Scientific.
 [10] D. P. Bertsekas (2001), Dynamic Programming and Optimal Control,
Volume 2, Athena Scientific.
 [11] R. S. Sutton, A. G. Barto (1998), Reinforcement
Learning: An Introduction. (Online copy).
 Adaptive control:

 [12] K. Åström and B. Wittenmark (1995), Adaptive Control,
AddisonWesley (2nd ed.).
 [13] S. Sastry and M. Bodson (1989), Adaptive Control: Stability,
Convergence, and Robustness. (Scanned PDF).
Potential topics of presentations include, but are not
limited to:
 Deterministic optimal control: Calculus of
variations, dynamic programming, HamiltonJacobiBellman
equations, LQregulators, numerical methods, approximation
methods, Hinfinity control, robust optimal control
 Stochastic optimal control: stochastic
HamiltonJacobiBellman equations, stochastic dynamic
programming, LQGregulators, optimal filtering methods,
certainty equivalent control, neighboring optimal control,
numerical and approximation methods
 Suboptimal and adaptive control: dual control, self
tuning regulators, model reference adaptive systems, recursive
parameter estimation, system identification
 Approximate dynamic programming: model predictive
control, policy iteration, Qlearning, reinforcement learning
 Classical control: pole placement design,
PID controllers, observers, connections between classical
and optimal/stochastic control
 Applications
Schedule
Date  Talker  Subject 
Download 
17.9.  Simo Särkkä 
Overview of the Topic
 PDF slides 
24.9.  Simo Särkkä 
Introduction to Classical, Optimal and Stochastic Control
 PDF slides 
1.10.  Tuomas Lukka 
Introduction to Adaptive and Learning Control
 PDF slides 
8.10.  Pasi Jylänki 
Linear Quadratic Regulator
 PDF 
15.10.  Jouni Hartikainen 
Dynamic Programming and HBJ Equation
 PDF 
22.10.  Antti YliKrekola 
Stochastic Dynamic Programming and SHJB Equation
 PDF 
29.10.  Sami Terho 
Classical Control
 PDF 
5.11.  
(no seminar)
 
12.11.  Jarno Vanhatalo 
Stochastic Linear Quadratic Regulator Problem and the Kalman Filter
 
19.11.  
(no seminar)
 
26.11.  
(no seminar)
 
3.12.  Sami Terho Jouni Hartikainen 
Reinforcement Learning
Classical Adaptive Control

PDF
PDF

10.12.  Antti YliKrekola
Pasi Jylänki
 Neural Networks in Control
Adaptive Dual Control 
PDF
PDF

General instructions
In every seminar talk, there is the author who writes a 24 page
summary paper about the subject and gives the talk, and an opponent,
whose task is to make questions and stimulate discussion after the
presentation. Every student is the author for (at least) one talk
and the opponent for another talk. The author should send the
summary paper to the opponent and to the teachers on the
thursday preceding the presentation day. The slides should
be sent to the teacher lastest on the day preceding the
presentation. The opponent prepares at least 23 questions about
the topic to stimulate discussion after the talk. Normal talk is
3040 minutes.
Homework exercises
Available here. DL 31.1.2008.
Exercise work
Available here. DL 29.2.2008.
Additional Online Material:
Contact:
When sending email concerning the course, please add "S114.4220" or
"1144220" to subject.
Dr.Tech. Simo Särkkä
ssarkka@lce.hut.fi
Ph.D. Tuomas Lukka
tjl@lce.hut.fi
Tästä sivusta vastaa
ssarkka@lce.hut.fi.
Sivua on viimeksi päivitetty 03.01.2008
URL:
http://www.lce.hut.fi/teaching/S114.4220/s2007/