CS 7650 - Natural Language Processing (Georgia Tech)
graduate-level NLP class at Georgia Tech
- Instructor: Wei Xu
- Lecture: Mondays, Wednesdays 2:00-3:15pm
Schedule (subject to change as the term progresses)
- Resource
- PyTorch Tutorial (w/ links to Colab notebook)
- Aug 18
- Course Overview
- Eisenstein 1
- Aug 20
- Machine Learning Review - linear classification
- Eisenstein 1, J+M 4
- Aug 21
- Problem Set 0 due
- Aug 25
- Machine Learning Review - logistic regression, perceptron, SVM
- Eisenstein 2.0-2.5, 4.1, 4.3-4.5, J+M B
- Aug 27
- Machine Learning Review - mutliclass classification
- Eisenstein 2.0-2.5, 4.1, 4.3-4.5, J+M 4
- Aug 29
- Project 0 due
- Sep 3
- Neural Networks - Feedforward, optimization
- Eisenstein 2.6, 3.1-3.3, J+M 6, Goldberg 1-4, J.G. Makin - Backpropagation
- Sep 5
- Problem Set 1 due
- Sep 8
- Word Embeddings
- Eisenstein 3.3.4, 14.5, 14.6, J+M 5, Goldberg 5
- Sep 10
- Sequence Models - HMM, Viterbi
- Eisenstein 7.0-7.4, J+M 17, J+M A
- Sep 15
- Conditional Random Fields
- Eisenstein 7.5, 8.3, J+M 17
- Sep 17
- Recurrent Neural Networks + Neural CRFs
- Eisenstein 3.4, 7.6, Goldberg 10-11, J+M 13
- Sep 19
- Project 1 due
- Sep 22
- Encoder-Decoder + Attention
- Eisenstein 18.3 - 18.5
- Sep 24
- No Class
- Sep 29
- Transformer
- J+M 8, Vaswani+17 Transformers, Alammar’s blog post, Rush’s tutorial
- Oct 1
- Convolutional Neural Networks, MT Evaluation
- Eisenstein 3.4, 7.6, Goldberg 9
- Oct 3
- Project 2 due
- Oct 8
- Pretrained Language Models - part 1 (ELMo, BERT & variants, BART/T5)
- J+M 10, ELMo BERT, BART
- Oct 10
- Course Project Proposal Due
- Oct 13
- Pretrained Language Models - part 2 (GPT2/3, knowledge distillation, instruction tuning)
- J+M 7, Hinton+15 Knowledge Distillation, GPT-3
- Oct 15
- Post-training of Language Models - part 3 (InstructGPT, PPO, decoding), Midterm Review
- InstructGPT
- Oct 21
- Slides for In-class Presentation due
- Oct 24
- Problem Set 2 due
- Nov 5
- In-class Midterm (close book, close note)
- Preview
- Open-source Language Models (part 4)
- Llama 3