| Category | Difficulty |
|---|---|
| HW | 3 |
| Exams | 3 |
18-847F is a special topics course in computer system: Foundations of Cloud and Machine learning infrastructure taught by professor Gauri Joshi. The objective of this course is to introduce students to modern cloud and machine learning infrastructure, and its theoretical foundations. It's a research paper reading based course, where the each lecture covers two paper, followed by assignments on the same. Overall the course provides you a system’s perspective of Machine Learning and cover leading research used for scaling ML systems.
As this is research paper based course, the topics change frequently depending on the professor. But generally following topics are covered:
- The first half of the course covers distributed computing and storage systems, frameworks such as MapReduce and Spark, and discuss scheduling and load balancing policies used in them.
- In the context of distributed storage systems, the course discusses coding-theoretic techniques used to improve availability and repair failed nodes. This section covered papers such as ‘Sparrow’, ‘Attack of clones’, ‘Rateless coding’, ‘Gradient coding’ etc.
- The second half of the course focuses on machine learning infrastructure in stochastic gradient descent and its implementation in large-scale systems coupled with adaptive communication strategies. Some of the papers covered where: ‘DistBielef’, ‘HolgWild!’, ‘Slow and stale gradients’, PipeDream’, ‘Cooperative SGD’, Adacomm’, ‘Federated learning’ etc.
- The third section was a bit exotic and covered wide range of SYSML topics ranging from model compression to hyperparamter tuning to multi-armed bandits, Gaussian processes and bayesian optimisation. This covered paper such as ‘TernGrad’, ‘ATOMO’, PowerSGD’, ‘HyperBand’, ‘Neural Architecture Search’, ‘Parallel Bayesian optimisation’ etc and guest lectures on Multi-armed bandits and Gaussian processes.
The course has 3 Exams/In-class-Quizzes, 6 HWs and class presentation on a research paper. The grading breakdown is as follows:
- 45% Homework
- 35% In-class Quizzes
- 10% Class Presentation(s)
- 10% Class Participation
It's not supposed to be an intensive course. The HW would not be intensive and would check if you grasped the main ideas of paper. Reading the papers would be sufficient, to clear quizzes/exams and HWs. The first few lectures of class, would cover concepts from probability and statistics relevant to the class, hence highly recommended that you attend the first few classes.
You can check out the previous year's material here: https://www.andrew.cmu.edu/course/18-847F/ You can check fall 2019's course syllabus here: https://www.andrew.cmu.edu/course/18-847F/syllabus_pdf.pdf