IFT6289, Winter 2022

Natural Language Processing with Deep Learning



Time and Place

Tuesdays 9:30am - 11:30am
Online via Zoom

Fridays 3:30pm - 5:30pm
Online via Zoom

Please check the Studium platform for the link to Zoom and Slack workspace of this course.
Note that the lectures will be recorded. The link to each recorded lecture will also be posted on Studium after class.

Instructor

Bang Liu
Office hours: Tuesdays 2:00pm - 4:00pm. (Online via Zoom/Slack workspace. Same meeting room with our lectures)
Email: bang.liu@umontreal.ca

Course description

Natural Language Processing (NLP) is a branch of artificial intelligence that deals with the interaction between computers and humans using natural language. It is one of the most important technologies of the information age and is used everywhere: search engines, advertising, chatbots, language translation, virtual agents, and so on. Deep Learning approaches have obtained very high performance across many different NLP tasks in recent years. In this course, students will gain a thorough introduction to the basics of NLP, as well as cutting-edge research in Deep Learning for NLP. We will focus on modern techniques for NLP, as well as introduce the applications in our daily lives. Students are encouraged to do some pretty cool research projects based on NLP techniques, e.g., writing poetry, detecting spam emails, building chatbots, machine reading comprehension, and so on. Through lectures, assignments, and a term project, students will learn the necessary skills to design, implement, and understand their own models for NLP tasks.

Prerequisites


Reference Texts

No textbook is required. But the following texts that can be read free online are helpful. The following book is helpful to give you more background about neural networks.

Marking scheme

Late policy:
A late day extends the deadline 24 hours. For ALL assignments, submissions after 2 late days (48 hours) of the deadline won't be accepted.
For programming assignments, we deduct 2% for each late day. We don't count hours, e.g., if you submit an assignment after 25 hours, it will be considered as 2 late days and will be deducted 4%.
For project proposal, midway report, we deduct 1% for each late day.
For project final report, we deduct 3% for each late day.
No late day for the final project presentation and reading assignments.

Tentative Time Table

Note: tentative schedule is subject to change.
Date Topic
Section I: Introduction / background
Lecture 1 (Jan 11) Introduction to NLP
Lecture 2 Basics of Deep Learning: Backpropagation and Neural Networks
Section II: NLP core techniques
Lecture 3 Language Modeling and Recurrent Neural Networks
Lecture 4 Word Meaning and Word Embedding
Lecture 5 Sentence Embeddings, Convolutional Neural Networks
Lecture 6 & 7 Graph Representations for NLP, Graph Convolutional Network
Lecture 8 Machine Translation, Seq2Seq and Attention
Lecture 9 Transformer and BERT
Lecture 10 Pre-trained Language Models (student mini lectures)
Lecture 11 Constituency Parsing
Lecture 12 Syntactic Dependency Parsing
Section III: Cutting-edge research topics.
Lecture 13 Data, Knowledge, and Logic: Modeling and Reasoning for Natural Language Understanding
Lecture 14 Guest lecture, TBD
Lecture 15 Knowledge Graph
Lecture 16 & 17 Conference tutorial, TBD
Lecture 18 & 19 Conference tutorial, TBD
Lecture 20 & 21 Conference tutorial, TBD
Lecture 22 & 23 Course project presentations and discussions.

Resources

Software