Course Info
Deep learning is a transformative technology that stands at the forefront of artificial intelligence research and applications. Its ability to learn from vast amounts of data and its versatility across a wide range of domains—from medical diagnosis to autonomous driving—makes it an essential skill for the future. As technologies evolve, deep learning provides the tools to not only enhance current systems but also to innovate and drive advancements that were once thought impossible. By mastering deep learning, students at Technion will be equipped with cutting-edge skills that are highly sought after in the job market, enabling them to contribute to significant technological impacts across industries worldwide.
This course will focus on the theory and algorithms behind deep learning, as well as on software techniques that allow efficient training of deep learning algorithms. It is a graduate-level course which provides both the necessary theoretical background and the hands-on experience required to be an effective deep learning practitioner, or to start on the path towards deep learning research.
Learning Outcomes
At the end of the course, the student will:
- Understand the key notions of deep learning, such as neural networks, learning regimes, optimization algorithms and training methodologies.
- Be able to apply deep learning algorithms to real-world data and problems.
- Know how to effectively use python and pytorch to implement models and algorithms from the recent literature.
- Perform a small research project using the studied notions and techniques.
Administration
Evaluation:
First Homework assignment : 10% Second Homework assignment : 25% Third Homework assignment : 25% Final Project: 40%
The Project might have a competitive component to it, but it will not effect your final grade by more then 5%.
Reserve Duty: Please come to the tutor in charge if you have specific needs due to army servise!
Late submission: In general, late submission will not be accepted, i.e. graded as 0. If you have a valid reason to request extention please contact the course staff.
Language: All course materials (including lecture and tutorial videos) are provided in English. This semester the frontal lectures and tutorials will be in English.
Credits: 3.0.
Prerequisites
This is an advanced course. Without both mathematical maturity and programming competency it will be very challenging to complete. The recommended pre-requisites are as follows:
- A good background of linear algebra, probability and calculus. See the supplemental material page if you need a refresher on one of these.
- Programming competency. The course will be very hands-on; much programming will be required. We will use Python exclusively, so it’s crucial to have experience with it.
- An introductory course about machine learning and/or signal/image processing.
Collaboration Policy and Honor Code
By enrolling in this course, you agree that you will strictly follow our collaboration policy as specified below. Any violation of this policy will result in an immediate failure in the course, and treatment by the Technion regulations committee.
- Submission of assignments is in singles or pairs. You are free to form study groups and discuss homeworks with other students. However, you must implement all required code independently of other groups (only with your submission partner).
- Submitted work must only be your own. You must do your own thinking, coding, debugging and write all answers yourself. We will run automatic plagiarism-detection software on your submissions to enforce this policy.
- You may not use any solutions from previous semesters’ homeworks.
- You may not share your solutions with other students.
- You may not upload your homework solutions to any public website, such as github. Private repos are OK, but they must remain so even after course completion.
Course Staff
The course staff:
Lecturers
TAs
Checkers
Literature
The course does not follow any specific book due to the rapid changes in the field. For your own reference, the following material may be useful.
-
Deep Learning
Ian Goodfellow, Yoshua Bengio, Aaron Courville
MIT Press, 2016
-
Deep Learning
Christopher M Bishop, Hush Bishop
Springer, 2024
Detailed Syllabus
The lectures and tutorials are In zoom only, and suplementry matirial with teoretical background exist in the Lectures page.
The tutorials are based on detailed and self-contained Jupyter notebooks, which guide you through.
The course also includes hands-on homework assignments in which you’ll implement working real-world models and run them on GPUs on the course servers. Performing the assignments by yourself is a crucial aspect of the course, which will provide you with many of the technical skills required to be effective with Deep Learning.
This semester’s syllabus is provided below.
| # | Date | Lecture | Tutorial | Homework |
|---|---|---|---|---|
| 1 | 11/11/2024 |
Introduction + Supervised learning | Supervised learning, PyTorch I | |
| 2 | 18/11/2024 |
Neural networks, CNNs I | MLP, PyTorch II | HW1 |
| 3 | 25/11/2024 |
Neural networks, CNNs II | CNNs | |
| 4 | 02/12/2024 |
Optimization and Training I | Optimization | |
| 5 | 09/12/2024 |
Optimization and Training II | Automatic diffrentiation | HW2 |
| 6 | 16/12/2024 |
Sequence models | Object detection | |
| 7 | 23/12/2024 |
Attention | RNN | |
| - | 30/12/2024 |
hanuka- no class | HW3 | |
| 8 | 06/01/2025 |
Transformers | Attention | |
| 9 | 13/01/2025 |
VIT | Transformers | |
| 10 | 20/01/2025 |
GAN | GANs | |
| 11 | 27/01/2025 |
Diffusion models | DDPM | Final Project |
| 12 | 03/01/2025 |
Multi-modality | Multi-modality |