| Category | Difficulty |
|---|---|
| HW | 4 |
| Exams | 8 |
| Project | 7 |
Natural Language Processing (NLP) is an exciting application field for machine learning, since NLP is found almost everywhere, from Siri on your phone, to telephone operators, automatic subtitle generation, and sentiment analysis for documents. Despite how hot NLP is, this class has not been known to be an instructive class, and is more about showing and introducing a bunch of concepts, rather than going in-depth to teach them. If you're doing research or you have a solid foundation in machine learning, this class may be okay for you. If you are a beginner at programming or don't have any machine learning experience, this class will be hard to keep up in and you will have a hard time learning things.
The lectures in this class are more of a way to show you a bunch of concepts, rather than teach you skills. Algorithms are glanced over, and you are expected to learn them on your own. In addition, any prerequisites you are just expected to know. Therefore, a lot of people find the lectures pretty dry, and hard to pick up things. My suggestion is that if you don't have some sort of machine learning background i.e. 10-601, then this class is probably not going to be very fun.
Lectures are recorded on Zoom and posted on Canvas, so you can watch the video back later when you want to complete the homework assignment.
Homework assignments are usually easy to medium length programming assignments that involve implementing a concept or algorithm learned in class. There are also sometimes short answer portions. Overall, the homeworks are not too hard. They may be slightly challenging without previous ML experience, but even then simply following the slides will be enough to complete the assignment. For a given homework assignment There is usually a lecture about a week before describing the concept/algorithm that you will need to implement for the homework. Sometimes the assignment can be as simple (or complicated) as translating a pseudocode function from the lecture slides.
Side note: a few homework assignments will be run on Google Colab notebooks, and there's a good chance you will run out of compute units as you work. Colab will prompt you to buy compute units, but you can just switch to another Google account and keep working, then switch back once that account is empty. That way you don't have to spend $200 on compute units (like one person did).
There is a semester-long project in the class where you have to design a Question and Answer generation program, where your program performs the two following functions:
Question(Article) => Questions about the articleAnswers(Article, Questions) => Answers to the questions in the article
The entire class competes to see who has the best question and answer program, where the class performs peer-review at the end of the semester.
You pick groups within the first two weeks of the semester, but even if you don't have friends taking the class, a lot of people will also be looking for groups. However, the project is where not having ML experience can really hurt. Several useful concepts aren't covered until later in the semester, and figuring out how to even approach the problem can be a struggle. Ideally you or someone in your group should have ML experience.
Project-related assignments include a literature search in the first few weeks of the semester, a proposal in the next couple weeks (very likely that your project will look nothing like it by the end), a progress report a couple weeks later (very possible that you still have nothing at this point), QA and QG (question answering/generation) prototype deadlines, and eventually the final system and report.
The assignment itself is pretty interesting, but the class poorly motivates the assignment. Most people struggle to figure out how to actually approach the problem. Usually, the way it goes is
- Beginning: I'm going to do a deep-learning semantically tagging based analysis, feed it through a multi-layer network and then perform reinforcement learning to refine my questions and answers.
- Checkpoint: Ok, to be more realistic, we will use a neural network to learn question and answer characteristics, and automate a training program to iteratively produce better questions and answers.
- Final: I'm just going to pattern match a bunch of templates to questions and answers
Surprisingly, you can generate pretty solid questions with just templates and some surrounding context. For example, the following program will already do pretty well
# Warning: this is just python pseudocode
def Questions(text):
questions = []
for sentence in text:
if sentence matches template '<phrase 1> is/was <phrase 2>':
questions.append('Who/what is <phase 1>?')
return questionsIf you just find enough templates, the quality of your final questions will be pretty solid. I know it seems cheap, but this is a weakness of the assignment, where an easy solution is very solid and can perform better than almost all complicated solutions.
Because most teams do not do very well on this assignment, grading is fairly lenient. Doing pattern-matching usually gets you around a B if you make it give reasonable questions and answers. TLDR, it's in your best interest to start with something pretty easy, and improve that if you have time.
Something else to consider is that the project is a sizeable percentage of your final grade (43.33%).
The exams are very hard for this class -- mostly because the class is not well taught. Multiple choice questions feel like class trivia, and the best way to study for them is just exposing yourself to a lot of NLP practice exams and trivia. Making your cheatsheet while copying down random facts on slides is a good way to prepare for some of these. The short and long answer portion of the exam is pretty straightforward, but is very hit or miss. What this means is that if there is an algorithm you don't know well, and they test it, then you're likely to lose a ton of points for that question. Given that they don't tend to focus on how algorithms work in class, you'll be on your own to learn many of the algorithms in this class.
Exams are open note - you can print out however many pages of notes you want. This could mean that you attend/watch lecture and take good notes to refer to for your exam. This could also mean that you print out all of the lecture slides and take them into the exam (like many people do).
Some links I found helpful for algorithms not covered well in class:
- CYK Algorithm: https://www.youtube.com/watch?v=v5JCMiWRdjQ
- Earley Parsing: https://www.youtube.com/watch?v=7UsVbTzx_54