Lectures

The lecture presentations and the accompanying material for each lecture is posted here.

Lecture 1: Introduction

Origins of deep learning, course goals, overview of machine-learning paradigms.

Lecture 2: Neural Networks

Linear and multilayer Perceptron, loss functions, activation functions, pooling, weight sharing, convolutional layers, gradient descent, backpropagation.

Lecture 3: Neural Networks II

advance architectures

Lecture 4: Optimization

Approximation, estimation and optimization errors, regularization, loss surface curvature, descent-based optimization methods, second-order methods.

Lecture 5: Training regime

Training regime for Deep Learning

Lecture 6: Object Detection

CV-based approaches, R-CNN, RPN, YOLO, SSD, losses, benchmarks and performance metrics.

Lecture 7: Sequence Models

RNN model, input-output sequences relationships, non-sequential input, layered RNN, backpropagation through time, word embeddings, attention, transformers.

Lecture 8: self supervision

Self supervision

Lecture 9: Attention

Attention mechanisem, transformers, self supervised encoder-decoder, LLM and generative models like GPT

Lecture 10: Vision transformers

Transformers in computer Vision, clip etc..

Lecture 11: Generative adverserial networks

Minimax and Nash equlibrium GAN, DCGAN, WGAN

Lecture 12: Variational AutoEncoders

variational inference, ELBO loss, reparametrization trick VAE, VQ-VAE and Dall-E

Lecture 13: Diffusion models

Diffusion process and models

Additional Resources

From previus semesters, the videos of Alex lectures
The supplemental material page contains prerequisite topics you should be familiar with.
Detailed notes will be available for most lectures on the lecture notes page.