Fridays 3:30pm - 5:30pm

Please check the Studium platform for the link to Zoom and Slack workspace of this course.

Note that

Office hours: Tuesdays 2:00pm - 4:00pm. (Online via Zoom/Slack workspace. Same meeting room with our lectures)

Email: bang.liu@umontreal.ca

- Proficiency in Python

All class assignments will be in Python (e.g., using NumPy and PyTorch). If you have a lot of programming experience but in a different language (e.g. C/C++/Matlab/Java/Javascript), you will probably be fine. - College Calculus, Linear Algebra

You should be comfortable taking (multivariable) derivatives and understanding matrix/vector notation and operations. - Basic Probability and Statistics

You should know the basics of probabilities, gaussian distributions, mean, standard deviation, etc. - Foundations of Machine Learning

If you already have basic machine learning and/or deep learning knowledge, the course will be easier.

- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning

- Jacob Eisenstein. Natural Language Processing

- Dan Jurafsky and James H. Martin. Speech and Language Processing (3rd ed. draft)

- Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing

- Michael A. Nielsen. Neural Networks and Deep Learning

- Readings (10%):

10 readings assignments worth 1% each - Assignments (45%):

3 programming assignments worth 15% each - Term project (45%):

Project proposal (up to 2 pages): 5%

Midway report (up to 4 pages): 5%

Final presentation: 5%

Final report (up to 8 pages): 30%

- Bonus: class participation (2%):

Students are encouraged to actively participate in the class, slack, etc.

A

For programming assignments, we deduct 2% for each late day. We don't count hours, e.g., if you submit an assignment after 25 hours, it will be considered as 2 late days and will be deducted 4%.

For project proposal, midway report, we deduct 1% for each late day.

For project final report, we deduct 3% for each late day.

No late day for the final project presentation and reading assignments.

Date | Topic |
||

Section I: Introduction / background | |||
---|---|---|---|

Lecture 1 (Jan 11) | Introduction to NLP | ||

Lecture 2 | Basics of Deep Learning: Backpropagation and Neural Networks | ||

Section II: NLP core techniques | |||

Lecture 3 | Language Modeling and Recurrent Neural Networks | ||

Lecture 4 | Word Meaning and Word Embedding | ||

Lecture 5 | Sentence Embeddings, Convolutional Neural Networks | ||

Lecture 6 & 7 | Graph Representations for NLP, Graph Convolutional Network | ||

Lecture 8 | Machine Translation, Seq2Seq and Attention | ||

Lecture 9 | Transformer and BERT | ||

Lecture 10 | Pre-trained Language Models (student mini lectures) | ||

Lecture 11 | Constituency Parsing | ||

Lecture 12 | Syntactic Dependency Parsing | ||

Section III: Cutting-edge research topics. | |||

Lecture 13 | Data, Knowledge, and Logic: Modeling and Reasoning for Natural Language Understanding | ||

Lecture 14 | Guest lecture, TBD | ||

Lecture 15 | Knowledge Graph | ||

Lecture 16 & 17 | Conference tutorial, TBD | ||

Lecture 18 & 19 | Conference tutorial, TBD | ||

Lecture 20 & 21 | Conference tutorial, TBD | ||

Lecture 22 & 23 | Course project presentations and discussions. |

- PyTorch an open-source deep learning library.
- DGL an open-source library for deep learning on graphs.
- HuggingFace Transformers an open-source library containing PyTorch and Tensorflow implementations, pre-trained model weights, usage scripts and conversion utilities for a variety of pre-trained language models.