Fundamentals of natural language processing, automatic speech recognition and speech synthesis; lab projects concentrating on building systems to process written and/or spoken language.
Date | Topic | Required Reading | Suggested Reading |
---|---|---|---|
1/11 | Course Overview | J+M, 2nd Edition Chapter 1 | |
1/13 | Probability Review and Naive Bayes | Mackay Book 2.1-2.3 (Probability), J+M, 3rd Edition 6.1 (Naive Bayes) | Crane's notes (Bayes' Rule) |
1/13 | Guest Speaker: Zhou Yu (CMU) Dreese Labs 480, 3:00pm | ||
1/18 | Text Classification | J+M, 3rd Edition 6.2-6.4 (Naive Bayes) | Shimodaira's notes (Naive Bayes) |
1/20 | Logistic Regression | J+M, 3rd Edition 7.1-7.3 (Logistic Regression) | Michael Collins' notes (Log-Linear Models) |
1/25 | More Logistic Regression | J+M, 3rd Edition 7.4-7.5 (Logistic Regression) | |
1/27 | Multi-class Logistic Regression and Perceptron | J+M, 3rd Edition 6.6-6.8 (Multi-class), Daume's CIML 4.1-4.4 (Perceptron Algorithm) | |
2/1 | More Perceptron | Daume's CIML 4.5-4.7 (Perceptron Algorithm) | |
2/3 | Language Modeling | J+M, 3rd Edition 4.1-4.2 (Language Models) | |
2/8 | More Language Modeling | J+M, 3rd Edition 4.3-4.4 (Language Models) | |
2/10 | Kneser-Ney Smoothing | J+M, 3rd Edition 4.4-4.6 (Language Models) | Michael Collins' notes (Language Models) |
2/15 | Parts of Speech and Hidden Markov Models | J+M, 3rd Edition 10.1-10.3 (Part-of-Speech Tagging) and 9.1-9.2 (Hidden Markov Models) | Michael Collins' notes (Hidden Markov Models) |
2/17 | The Viterbi Algorithm | J+M, 3rd Edition 9.3-9.4, 10.4 | |
2/22 | Maximum Entropy Markov Models | J+M, 3rd Edition 10.5 | |
2/24 | Log-linear Models | J+M, 3rd Edition 7.1-7.5 (Logistic Regression) | Michael Collins' notes (Log-Linear Models) |
3/1 | More Maximum Entropy Markov Models | J+M, 3rd Edition 10.5 | Michael Collins' notes (MEMMs) |
3/3 | Midterm Review | ||
3/3 | Distinguished Speaker: Raymond J. Mooney (UT Austin) Dreese Labs 480, 3:00pm | LSTM for Language and Vision | Venugopalan et al.'s paper (Vedio Captioning) |
3/8 | Guest Lecture: Jeniya Tabassum | Probabilistic Graphical Model with Latent Variables | Tabassum et al.'s EMNLP 2016 paper |
3/10 | Midterm (in class, closed book) | ||
3/22 | Conditional Random Fields and Structured Perceptron | Daume's CIML 17.1-17.3 and 17.6-17.7 (Structured Prediction) | Sutton and McCallum's CRF Tutorial |
3/24 | Syntax and Context Free Grammars | J+M, 3rd Edition Chapter 11 (Formal Grammars) | |
3/24 | Guest Speaker: Dan Garrette (Google) Dreese Labs 480, 3:00pm | Combinatory Categorial Grammars (CCGs) | |
3/29 | Parsing and CKY Algorithm | J+M, 3rd Edition Chapter 12 (Syntactic Parsing) | Collins et al.'s paper (Global Linear Model) |
3/31 | Automatic Speech Recognition | J+M, 2nd Edition Chapter 9 (Automatic Speech Recognition) | Juang and Rabiner's Automatic Speech Recognition – A Brief History of the Technology Development, Gales and Young review (HMM-based ASR) |
4/5 | Guest Lecture: Micha Elsner | Integer Linear Programming (ILP) | Clarke and Lapata's paper (Integer Linear Programming) |
4/7 | Deep Learning for Speech Recognition | Graves et al.'s paper (Connectionist Temporal Classification) | |
4/12 | More Deep Learning for Speech Recognition | Baidu Research's paper (Deep Speech) | |
4/14 | Neural Network Language Modeling | Koehn's 15.1-15.2.3 (Neural Networks) | Bengio et al.'s paper (Neural Language Models) |
4/19 | Deep Learning for NLP | Goldberg's tutorial (Neural Networks in NLP) | |
4/21 | Project Presentations |