Link Search Menu Expand Document

CS 4650 - Natural Language Processing (Georgia Tech)

  • Instructor: Wei Xu
  • Lecture: Mondays, Wednesdays 2:00-3:15pm

Schedule (subject to change as the term progresses)

Resource
PyTorch Tutorial (w/ links to Colab notebook)
Jan 6
Course Overview
Eisenstein 1
Jan 8
Machine Learning Review - linear classification (notes)
Eisenstein 1, J+M 4
Jan 9
Problem Set 0 due
Jan 13
Machine Learning Review - logistic regression, perceptron, SVM (notes)
Eisenstein 2.0-2.5, 4.1, 4.3-4.5, J+M 5
Jan 15
Machine Learning Review - mutliclass classification
Eisenstein 2.0-2.5, 4.1, 4.3-4.5, J+M 5
Jan 17
Project 0 due
Jan 27
Neural Networks - Feedforward, optimization (notes)
Eisenstein 2.6, 3.1-3.3, J+M 7, Goldberg 1-4, J.G. Makin - Backpropagation
Jan 29
Word Embeddings
Eisenstein 3.3.4, 14.5, 14.6, J+M 6, Goldberg 5
Jan 30
Problem Set 1 due
Feb 3
Sequence Models - HMM, Viterbi (notes)
Eisenstein 7.0-7.4, J+M 17, J+M A
Feb 5
Conditional Random Fields (notes)
Eisenstein 7.5, 8.3
Feb 7
Project 1 due
Feb 10
Recurrent Neural Networks (notes)
Eisenstein 7.6, Goldberg 10-11, J+M 8
Feb 12
Guest Lecture - Multilingual Multi-Cultural LLMs - by Tarek Naous
Feb 17
Convolutional Neural Networks + Neural CRFs
Eisenstein 3.4, 7.6, Goldberg 9
Feb 19
Encoder-Decoder + MT
Eisenstein 18.3 - 18.5
Feb 21
Project 2 due
Feb 24
Attention+ Tokenization (notes)
Wu+16 Google NMT, Holtzman+19 Degeneration
Feb 26
Transformer (notes)
J+M 9, Vaswani+17 Transformers, Alammar’s blog post, Rush’s tutorial
Mar 3
Pretrained Language Models (part 1), Midterm Review
J+M 11, ELMo BERT
Mar 4
Problem Set 2 due
Mar 5
Pretrained Language Models (part 2) + Ethics
J+M 10, BART, GPT-3
Mar 10
Post-training of Language Models (part 3)
InstructGPT
Mar 12
Open-source Language Models (part 4)
Llama 3
Mar 24
Project Proposal due
Mar 26
Guest Lecture - Evaluation of LLM-generated Text - by Yao Dou
Mar 28
Project 3 due
Apr 7
In-class Midterm (practice exam)