Lectures
The accompanying material for each lecture is posted here.
Lecture 1: Introduction
Origins of deep learning, course goals, overview of machinelearning paradigms, intro to computational acceleration.
Lecture 2: Supervised learning
Supervised learning problem statement, data sets, hypothesis classes, loss functions, basic examples of supervised machine learning models, adding nonlinear...
Lecture 3: Neural Networks
Linear and multilayer Perceptron, loss functions, activation functions, pooling, weight sharing, convolutional layers, gradient descent, backpropagation.
Lecture 4: Training Neural Networks
Approximation, estimation and optimization errors, regularization, loss surface curvature, descentbased optimization methods, secondorder methods.
Lecture 5: Sequence Models
RNN model, inputoutput sequences relationships, nonsequential input, layered RNN, backpropagation through time, word embeddings, attention, transformers.
Lecture 6: Unsupervised Learning and Generative Models
Subspace models, autoencoders, unsupervised loss, generative adversarial nets, domain adaptation.
Lecture 7: Reinforcement Learning
Markov decision process, policies, rewards, value functions, the Bellman equation, qlearning, policy learning, actorcritic learning, AutoML.
Lecture 8: Intro to Parallel Architectures
Parallelism and multithreading, execution of DL tasks, CPU vs. GPU architectures, CUDA programming model.
Lecture 9: Parallel Architectures for Training
NVidia GPU architectures, memory hierarchy, CUDA threads, unified memory, optimizations for CNNs, hardware architectures for training.
Lecture 10: Parallel Architectures for Inference
Challenges of inference, lowbit representations, pruning, GPU vs FPGA and ASIC, TPU architecture.
Lecture 11: Learning on NonEuclidean Domains
Toeplitz operators, graphs, fields, gradients, divergence, LaplaceBeltrami operator, noneuclidean convolution, spectral and spatial CNN for graphs.
Lecture 12: Neural network compression
SVD for linear layers, flattened and grouped convolutions, pruning and retraining, network weights quantization.
Lecture 13: Object detection
CVbased approaches, RCNN, RPN, YOLO, SSD, losses, benchmarks and performance metrics.
Additional Resources

The supplemental material page contains prerequisite topics you should be familiar with.

Detailed notes will be available for most lectures on the lecture notes page.

You can browse the course youtube playlist from the current and previous semester. New videos will be posted on the site.