IFT 6760A - Winter 2021
Matrix and tensor factorization techniques for machine learning
[fr]
- The first class will be on Thursday January 14th at 11:30
- The page of last year's edition of the class is available here for references.
Description
The goal of this course is to present an overview of linear and multilinear algebra techniques for designing/analyzing ML algorithms and models, and to engage students with new research in the area.
- Fundamental notions of linear and multilinear algebra.
- Old and new ML methods leveraging matrix and tensor decomposition: PCA/CCA,
collaborative filtering, spectral graph clustering, spectral methods for HMM, K-FAC,
spectral normalization,
tensor method of moments, NN/MRF compression, tensor regression/completion, etc.
- Open problems.
Class Info
When:
Tuesdays 12:30-2:30PM and Thursdays 11:30-1:30PM
First day of class: Thursday January 14th
Where: Online (zoom)
Instructor: Guillaume Rabusseau
Office hours: Tuesdays after class
Evaluation
Class project (30%) + Assignments (30%) + Quizzes (10%) + Paper presentation (30%)
Tentative list of topics (to be updated as we go along)
- Linear algebra fundamentals
- Vector spaces, linear maps, matrices, rank, change of basis, Hilbert spaces (orthogonality), etc.
- Matrix decomposition: eigenvalues/eigenvectors, diagonalizability (Jordan canonical form), Gram-Schmidt algorithm, QR decomposition, singular value decomposition.
- Linear algebra and matrix factorization techniques for ML
- Linear algebra is everywhere: PCA, CCA, LLE, ...
- Spectral graph clustering
- Spectral learning of HMMs, PCFGs and related models
- Collaborative filtering, matrix completion and low rank matrix recovery
- Multilinear algebra and tensor factorization techniques for ML: Tensors are the new matrices!
- Presentation of several tensor decomposition formats: CanDecomp/ParaFac (CP), Tucker, Tensor Train (TT), etc.
- Tensor networks: a unifying tool
- Some tensor decomposition algorithms: alternating least squares, SVD based algorithms (HOSVD, TT-SVD), DMRG like algorithms, etc.
- Tensor method of moments: consistent estimators from observable tensors
- Algorithms for CP decomposition of symmetric tensors: Jenrich's algorithm, tensor power method, sum of squares hierarchy...
- Multilinear extensions: tensor completion, tensor recovery, tensor regression, ...
- Compressing ML models (e.g. NN, MRF) with tensor factorization