Dynamical Systems and Control - 76929

Lecture notes, slides and relevant material (thanks to Lavi Shpigelman)

Schedule & Handouts:
date Subject notes Relevant material
19.10.09 Introduction and math review, intro to linear systems ppt pdf C-Leg_berlin_short.mpg
C-Leg_slope_warped5.avi
stand_up1.mov
stand_up2.mov
Bionic Arms
Morimoto and Doya 2001
Wolpert-lab, TOPS project demo
Harris & Wolpert 1998
Wolpert & Ghahramani 2000
26.10.09 Phase plane analysis, singular points,
linearization near stationary points
slides (ppt) Laplace Tutorial
(1st and) 2nd order LTI system analysis
2.11.09 Simple PID control near stationary points. slides(pdf)
notes (pdf)
numeric integration movie
22.11.09 Autonomous LTI systems in matrix form: solution as matrix exponential, stability, discrete time counterpart slides (pdf)
notes (pdf)
laplace table
Autonomous linear dynamical systems
Solution via Laplace transform and matrix exponential
Eigenvectors and diagonalization
LTI systems & Laplace: chen 2.1-2.3
Linear Algebra: Chen, 3, Stengel, 2.1-2.2
22.11.09 LTI systems with inputs: transfer functions, feedback notes (pdf)
Constrained optimization (Lagrange multipliers) slides (pdf)
notes (pdf)
Block Diagram reduction, Signal Flow Graphs, Mason's rule
Constrained optimization (Lagrange multipliers)
Lecture slides (pdf, 4up)
Block Diagram, SFG, Mason (10mb, pdf, acrobat 7)
Controllability slides (pdf)
notes (pdf)
Stengel: Chapter 2.5
Chen: 4.1, 4.2, 4.2.1, 6.1-6.3
Controllability (Boyd)
Least squares methods, SVD, pseudo inverse lecture slides
notes( ps / pdf )
Least squares methods (Boyd)
SVD (Boyd)
SVD continued, Observability, State estimate, Gaussian noise lecture slides
Notes(from last year. inc. Kalman Filter, not covered yet) ps / pdf )
Kalman Filter & EKF slides (by Stephan Boyd): estimation, Kalman filter, EKF
Stengel: Ch 2.4 (pp. 86-114) Ch 4, 4.1, 4.2 (pp 299-322), Ch 4.3 (pp 342-347)
Emery Brown's mouse tracking
KF on Wikipedia
Particle Filtering ps / pdf
slides(pdf)
A tutorial on particle filtering
(Arulampalam et al.)


Introduction to Monte Carlo methods (Mackay)

PF on Wikipedia
Intro to variational methods Bryson&Ho PP. 1-55
Variational mathods intro continued + Descrete time optimal control, LQR. ps / pdf LQR via Lagrange multipliers (Boyd)
LQR discrete time, finite horizon (Boyd)
yet more variants of variational methods in optimal control. slides
ps / pdf
Bryson & Ho, Chap 2-3 (see slides for specific sections)
HJB, Discrete version, Dynamic Programing, DP -> RL slides
Bryson & Ho Chap 4.
Inro to RL: DP, Monte Carlo ps / pdf
DP: Sutton & Barto, Chap. 4
Monte Carlo: Sutton & Barto, Chap. 5
ppt
Sutton and Barto,Reinforcement learning, an Introduction (the book, at Sutton's web site)
Kaelbling,Littman & Moore,Reinforcement Learning: A Survey,JAIR 1996
Intro to RL: TD(0) (Temporal Difference), Q-Learning Sutton & Barto, Chap. 6, Sections 1-6,10,11 W. Schultz,Predictive Reward Signal of Dopamin Neurons
RL - TD(lambda) , RL in continuous space/time TD slides(ppt)
Doya slides1(pdf)
Sutton & Barto, Chap. 7, Sec. 1-5
K. Doya, Neural Computation 12,2000

Last year's notes

Schedule & Handouts:
date teacher Subject notes Relevant material
Lavi EM for linear dynamical systems ps / pdf
slides(pdf)
A View of the EM Algorithm That Justifies Incremental, Sparse and other Variants (Neal and Hinton)

Parameter estimation for LDS (Ghahramani and Hinton)
7.1.07 Lavi Deriving system dynamics using Lagrange's Equations slides