Multilinear Systems and Invariant
Theory in the Context of Computer Vision and Graphics
CS329
The course is designed to teach the basic ideas behind linear
multi-dimensional mappings from high to lower dimensional
spaces. These techniques are widely used in 3D modeling of rigid
scenes in computer vision and graphics, some aspects of 3D visual
recognition, and modeling of non-rigid point configurations
applicable to modeling ``actions'' and ``behaviors''.
The lectures cover three parts:
-
P^2 -> P^2: (3 lectures) Two dimensional alignment problems
(image-to-image) of rigid and
non-rigid configurations. This includes the homography matrix,
infinitesimal direct estimation and
homography tensor, as well as some introductory material on projective
geometry and tensor notations.
-
P^3 -> P^2: (3 lectures) The reconstruction of three dimensions from
multiple 2D projections. This includes the plane+parallax model,
fundamental matrix, trifocal tensor, and shape tensors.
-
P^k -> P^2: (1 lecture) Non-rigid configurations re-casted as rigid
configurations in higher dimensional spaces. We will cover only the
case of P^4->P^2.
Prerequisites:
Minimal background of basic Linear Algebra is
recommended (you need
to know what a matrix is and some basic properties), all the rest
would be covered in class.
Students will be given 3 home assignments (theoretical problems) and
will have the option to choose between a final exam or project.
Teaching Staff:
Principal Instructor: Amnon Shashua
(shashua@cs.stanford.edu)
Send in your questions to shashua@cs.stanford.edu
The class will meet Mondays 9:00 - 11:00 in Gates
159.
The first class will be on Monday, January 14.
All announcements will be made through this class website. All students
enrolling in the course are expected to sign up to the
cs329@lists.stanford.edu mailing list by sending a mail to
majordomo@lists.stanford.edu with a blank subject field and the body
being subscribe cs329.
Course Content by Lecture (Syllabus)
-
Part I: P^2 -> P^2
-
Class 1: Projection matrices, basics of projective geometry of the plane, the
2D collineation (homography matrix) and its properties (derivation,
plane at infinity, the rank-4 constraint).
- Class 2: Tensor Product, covariant-contravariant tensor notations, Dynamic
P^2 -> P^2 alignment (Homography Tensor). Further reading material: Shashua-Wolf:ECCV'00.
- Class 3: Infinitesimal motion model of the plane, the constant brightness
equation and gradient-based estimation, direct estimation,
factorization principle.
-
Part II: P^3 -> P^2
- Class
4: The plane+parallax model, fundamental matrix, constraints across
3 views (why F-mats are not sufficient?), the trifocal tensor, slicing
properties.
- Class
5: Self Calibration: the basic equations, absolute conic, Kruppa's
euqations, recovering internal parameters. Further reading material: Hartley-Zisserman pages 441--479.
- Class
6: Duality and Shape tensors. Further reading material: Levin-Shashua:ECCV02.
-
Part III: P^k -> P^2
-
Non-rigid configurations re-casted as rigid
configurations in higher dimensional spaces where k=4,5,6. We will
cover in detail the geometry of
P^4->P^2.