Deep learning is widely used in a growing range of applications ranging from image classification and generation, text comprehension, signal processing, game playing and more. This course will focus on algorithms, programming frameworks and new hardware and software interfaces that aim to allow execution of deep learning algorithms in an efficient way. It will provide both the necessary theoretical background and the hands-on experience required to be an effective deep learning practitioner, or to start on the path towards deep learning research.
At the end of the course, the student will:
- Understand and be able to apply notions in deep learning.
- Know how to effectively use leading python machine-learning and deep learning frameworks such as PyTorch.
- Know how to optimize software and hardware performance in deep neural network applications.
- Know how to leverage GPUs and write custom computational kernels to accelerate both training and inference.
- Perform a small research project using the studied notions and techniques.
Evaluation: 40% Homework assignments, 60% final project.
Language: The course will be taught in English.
The course does not follow any specific book. For your own reference, the following material may be useful.
This semester, the course will be presented using a flipped-classroom approach.
Students are expected to watch and read the pre-requisite material, available from the couse Lectures page before each class. The in-class lectures will then be divided into a supplementary part, relating to the pre-requisite material and an introductory part presenting new material relating to the next lecture.
||1||Introduction to machine learning||Python, numpy and friends|
||2||Lecture 1b||Introductory: Supervised learning, probability and statistics||Logistic regression||HW1|
||3||Lecture 2||Supplementary: performance evaluation, ROC, confusion matrix;
Introductory: neural networks
||4||Lecture 3||Supplementary: CNNs architectures
Introductory: training, calculus, optimization
||5||Lecture 4||Training deep networks: Optimization, generalization and regularization||HW2|
||7||Lecture 5||Supplementary: Word embeddings, attention
Introductory: Unsupervised learning
||8||Lecture 6||Supplementary: GANs, image generation, domain adaptation
Introductory: Reinforcement learning
||9||Lecture 7||Supplementary: Actor-critic, AutoML, NAS
Introductory: Non-euclidean domains, harmonic analysis
||10||Lecture 11||Supplementary: Applications of CNNs on graphs
Introductory: Hardware accelerators
|Deep reinforcement learning|
||11||Lecture 8||Neural network compression and pruning|
||12||Lecture 9||Supplementary: GPU architectures||Geometric deep learning|
||14||Lecture 10||Supplementary: Hardware for inference||CUDA|