Course Info
Deep learning is widely used in a growing range of applications ranging from image classification and generation, text comprehension, signal processing, game playing and more. This course will focus on algorithms, programming frameworks and new hardware and software interfaces that aim to allow execution of deep learning algorithms in an efficient way. It will provide both the necessary theoretical background and the hands-on experience required to be an effective deep learning practitioner, or to start on the path towards deep learning research.
Learning Outcomes
At the end of the course, the student will:
- Understand and be able to apply notions in deep learning.
- Know how to effectively use leading python machine-learning and deep learning frameworks such as PyTorch.
- Know how to optimize software and hardware performance in deep neural network applications.
- Know how to leverage GPUs and write custom computational kernels to accelerate both training and inference.
- Perform a small research project using the studied notions and techniques.
Administration
Evaluation: 40% Homework assignments, 60% final project.
Language: The course will be taught in English.
Credits: 3.0.
Course Staff
Lecturers
TAs
Checkers
Literature
The course does not follow any specific book. For your own reference, the following material may be useful.
Detailed Syllabus
This semester, the course will be presented using a flipped-classroom approach.
Students are expected to watch and read the pre-requisite material, available from the couse Lectures page before each class. The in-class lectures will then be divided into a supplementary part, relating to the pre-requisite material and an introductory part presenting new material relating to the next lecture.
Date | # | Pre requisite | Lecture | Tutorial | Homework |
---|---|---|---|---|---|
17/03/2019 |
1 | Introduction to machine learning | Python, numpy and friends | ||
24/03/2019 |
2 | Lecture 1b | Introductory: Supervised learning, probability and statistics | Logistic regression | HW1 |
31/03/2019 |
3 | Lecture 2 | Supplementary: performance evaluation, ROC, confusion matrix; Introductory: neural networks |
MLP | |
07/04/2019 |
4 | Lecture 3 | Supplementary: CNNs architectures Introductory: training, calculus, optimization |
CNNs | |
14/04/2019 |
5 | Lecture 4 | Training deep networks: Optimization, generalization and regularization | HW2 | |
21/04/2019 |
6 | No class | |||
28/04/2019 |
7 | Lecture 5 | Supplementary: Word embeddings, attention Introductory: Unsupervised learning |
RNNs | |
05/05/2019 |
8 | Lecture 6 | Supplementary: GANs, image generation, domain adaptation Introductory: Reinforcement learning |
Domain adaptation | |
12/05/2019 |
9 | Lecture 7 | Supplementary: Actor-critic, AutoML, NAS Introductory: Non-euclidean domains, harmonic analysis |
HW3 | |
19/05/2019 |
10 | Lecture 11 | Supplementary: Applications of CNNs on graphs Introductory: Hardware accelerators |
Deep reinforcement learning | |
26/05/2019 |
11 | Lecture 8 | Neural network compression and pruning | ||
02/06/2019 |
12 | Lecture 9 | Supplementary: GPU architectures | Geometric deep learning | |
09/06/2019 |
13 | No class | HW4 | ||
16/06/2019 |
14 | Lecture 10 | Supplementary: Hardware for inference | CUDA | |
23/06/2019 |
15 | Project Presentations |