# IFT 6760A - Winter 2022

Matrix and tensor factorization techniques for machine learning

[fr]
- The first class will be on Tuesday January 11th at 12:30
- The page of last 2020's edition of the class (with lecture notes and bibliography) is available here for references.

## Description

The goal of this course is to present an overview of linear and multilinear algebra techniques for designing/analyzing ML algorithms and models, and to engage students with new research in the area.

- Fundamental notions of linear and multilinear algebra.

- Old and new ML methods leveraging matrix and tensor decomposition: PCA/CCA,
collaborative filtering, spectral graph clustering, spectral methods for HMM, K-FAC,
spectral normalization,
tensor method of moments, NN/MRF compression, tensor regression/completion, etc.

- Open problems.

## Class Info

**When:**
Tuesdays 12:30-2:30PM and Thursdays 11:30-1:30PM

**First day of class:** Tuesday January 11th

**Where:** Online until Jan. 17th (then TBD, either online or at Mila)

**Instructor:** Guillaume Rabusseau

**Office hours:** Tuesdays after class
## Evaluation

Class project (30%) + Assignments (30%) + Quizzes (10%) + Paper presentation (30%)

## Tentative list of topics (to be updated as we go along)

- Linear algebra fundamentals
- Vector spaces, linear maps, matrices, rank, change of basis, Hilbert spaces (orthogonality), etc.
- Matrix decomposition: eigenvalues/eigenvectors, diagonalizability (Jordan canonical form), Gram-Schmidt algorithm, QR decomposition, singular value decomposition.

- Linear algebra and matrix factorization techniques for ML
- Linear algebra is everywhere: PCA, CCA, LLE, ...
- Spectral graph clustering
- Spectral learning of HMMs, PCFGs and related models
- Collaborative filtering, matrix completion and low rank matrix recovery

- Multilinear algebra and tensor factorization techniques for ML:
**Tensors are the new matrices!**
- Presentation of several tensor decomposition formats: CanDecomp/ParaFac (CP), Tucker, Tensor Train (TT), etc.
- Tensor networks: a unifying tool
- Some tensor decomposition algorithms: alternating least squares, SVD based algorithms (HOSVD, TT-SVD), DMRG like algorithms, etc.
- Tensor method of moments: consistent estimators from observable tensors
- Algorithms for CP decomposition of symmetric tensors: Jenrich's algorithm, tensor power method, sum of squares hierarchy...
- Multilinear extensions: tensor completion, tensor recovery, tensor regression, ...
- Compressing ML models (e.g. NN, MRF) with tensor factorization