Last year version: Winter 2023
Structured prediction is the problem of learning a prediction mapping between inputs and structured outputs, i.e. outputs that are made of interrelated parts often subject to constraints. Examples include predicting trees, orderings, alignments, etc., and appear in many applications from computer vision, natural language processing and computational biology among others.
This is an advanced machine learning course that will focus on the fundamental principles and related tools for structured prediction. It will also present several advanced optimization concepts along the way, as they tend to be useful for structured prediction. The course will review the state of the art, tie older and newer approaches together, as well as identify open questions. It will consist of a mix of faculty lectures as well as a research project conducted by the students.
Prerequisite: I will assume that most of the content of IFT 6269 Probabilistic Graphical Models is known by the students.
Teacher: Simon Lacoste-Julien
Office: @Mila: D-2100 and @UdeM: 3339 André-Aisenstadt
Winter 2024 semester:
Classes are held at Mila Auditorium 1 – 2nd floor of 6650 Saint-Urbain Street
Tuesdays and Thursdays from: 09h30-11h30
First lecture is Tuesday January 9th; last lecture is April 11th; poster presentation is April 30th
Language of teaching will be English.
Project (80%) - project report to hand in + project presentation on April 30th 09:30am-11:30am | detailed info
Class participation (20%) - ask questions during class or discuss the course content on Slack. Potentially could also scribe a lecture.
Lecture notes + references for 2024 (updated along the way).
energy-based models & surrogate losses
theory of structured prediction: consistency, etc.
the generative learning / discriminative learning continuum
conditional random field (CRF)
structured SVM
latent variable structured SVM, CCCP algorithm
large-scale optimization: Frank-Wolfe, variance-reduced SGD, block-coordinate methods
learning to search
RNN & friends
combinatorial algorithms: min-cost network flow, submodular optimization, dynamic programs