This course presents the foundations of Deep Learning-based Natural Language Processing and Large Language Models. Topics addressed include recurrent neural networks, the attention mechanism and transformer architectures.
The lectures and lab projects will be presented by the following instructors:
Fanny Jourdan is a researcher at IRT Saint Exupéry working on the DEEL projet. Her research interests include Explainability and Evaluation of Natural Language Processing algorithms.
Mouhcine Mendil is a research engineer at IRT Saint Exupéry where he contributes to various projects, including DEEL and Hecate. His work focuses on Physics-Informed Machine Learning and Uncertainty Quantification.
Joseba Dalmau is a research engineer at IRT Saint Exupéry working on the DEEL projet and a part-time associate lecturer at INSA Toulouse. His research interests include Uncertainty Quantification for Machine Learning and Deep Learning models and Anomaly and Out-of-Distribution data detection.
By the end of the course, the students should be able to:
- Distinguish between the different NLP tasks.
- Identify the appropriate model architectures for solving each NLP tasks: RNN, Transformer.
- Implement, train and deploy the most common NLP models.
- Evaluate the results of different models and compare their performance through benchmarking experiments.
| Lecture | Date | Time | Room | Topic | Instructor |
|---|---|---|---|---|---|
| Lecture 1 | Jan 7 | 10h15-12h00 | B206 (N7) | NLP tasks, pre-processing | F. Jourdan |
| Lecture 2 | Jan 10 | 14h00-15h45 | B209 (N7) | Text vectorization | F. Jourdan |
| Lecture 3 | Jan 20 | 10h15-12h00 | B206 (N7) | ML techniques for NLP | M. Mendil |
| Lab Project 1 | Jan 20 | 14h00-15h45 | C305 (N7) | M. Mendil | |
| Lab Project 2 | Jan 20 | 16h15-18h00 | C305 (N7) | M. Mendil | |
| Lecture 4 | Jan 31 | 14h00-15h45 | B206 (N7) | DL techniques for NLP | J. Dalmau |
| Lab Project 3 | Jan 31 | 16h15-18h00 | C305 (N7) | J. Dalmau | |
| Lecture 5 | Feb 5 | 8h00-9h45 | B206 (N7) | Transformers | J. Dalmau |
| Lab Project 4 | Feb 5 | 10h15-12h00 | C305 (N7) | J. Dalmau | |
| Lecture 6 | Feb 12 | 14h00-15h45 | B206 (N7) | Feature spaces | J. Dalmau |
| Lab Project 5 | Feb 12 | 16h15-18h00 | C304 (N7) | J. Dalmau | |
| Exam | Mar 3 | 14h00-15h45 | C302 (N7) |
The only assessment for this course will be a written exam which will take place on March 3, 14h00-15h45. The instructors will provide the students with sample exercises so that they can train themselves for the exam. The students will be allowed a single A4 two-sided handwritten sheet of paper with the notes they deem useful for the exam.
Open-answer exercises will be evaluated according to the following criteria (if they apply):
| Category | 😀😀😀 | 🙂🙂🙂 | 😟😟😟 | 😭😭😭 |
|---|---|---|---|---|
| Result | The result is correct | The result is incorrect, but not contradictory with the formulation | The result is contradictory with the formulation | The result is not given |
| Justification | The justification is complete and flawless | The steps of the justification are the correct ones, but there are minor mistakes | Some steps of the justification are either missing or useless | No justification is given |
| Completion | All the necessary details for the understanding of the solution are given | Many details for the understanding of the solution are given | A few details for the understanding of the solution are given | No details for the understanding of the solution are given |
| Conciseness | No unnecessary details for the understanding of the solution are given | A few unnecessary details for the understanding of the solution are given | Many unnecessary details for the understanding of the solution are given | We are under the impression the solution applies to a totally different problem |
| Neatness | The solution is neat and properly organized. The steps of the justification as well as the answer are easily found | Some of the steps of the justification OR the solution are hard to find | Some of the steps of the justification AND the solution are hard to find | The solution is totally chaotic, looks like a scratch paper, or there is no solution |