Lectures
The lecture presentations and the accompanying material for each lecture is posted here.
Lecture 1: Introduction
Origins of deep learning, course goals, overview of machine-learning paradigms.
Lecture 2: Neural Networks
Linear and multilayer Perceptron, loss functions, activation functions, pooling, weight sharing, convolutional layers, gradient descent, backpropagation.
Lecture 3: Neural Networks II
advance architectures
Lecture 4: Optimization
Approximation, estimation and optimization errors, regularization, loss surface curvature, descent-based optimization methods, second-order methods.
Lecture 5: Training regime
Training regime for Deep Learning
Lecture 6: Object Detection
CV-based approaches, R-CNN, RPN, YOLO, SSD, losses, benchmarks and performance metrics.
Lecture 7: Sequence Models
RNN model, input-output sequences relationships, non-sequential input, layered RNN, backpropagation through time, word embeddings, attention, transformers.
Lecture 8: self supervision
Self supervision
Lecture 9: Attention
Attention mechanisem, transformers, self supervised encoder-decoder, LLM and generative models like GPT
Lecture 10: Vision transformers
Transformers in computer Vision, clip etc..
Lecture 11: Generative adverserial networks
Minimax and Nash equlibrium GAN, DCGAN, WGAN
Lecture 12: Variational AutoEncoders
variational inference, ELBO loss, reparametrization trick VAE, VQ-VAE and Dall-E
Lecture 13: Diffusion models
Diffusion process and models
Additional Resources
-
From previus semesters, the videos of Alex lectures
-
The supplemental material page contains prerequisite topics you should be familiar with.
-
Detailed notes will be available for most lectures on the lecture notes page.