A deep understanding of deep learning

1 How to learn from this course
2 Using Udemy like a pro

1 Downloading and using the code
1. DUDL_PythonCode
2 My policy on code sharing

1 What is an artificial neural network
2 How models learn
3 The role of DL in science and knowledge
4 Running experiments to understand DL
5 Are artificial neurons like biological neurons

1 Should you watch the Python tutorial

1. PyTorch or TensorFlow
2 Introduction to this section
3 Spectral theories in mathematics
4 Terms and datatypes in math and computers
5 Converting reality to numbers
6 Vector and matrix transpose
7 OMG its the dot product
8 Matrix multiplication
9 Softmax
10 Logarithms
11 Entropy and cross entropy
12 Minmax and argminargmax
13 Mean and variance
14 Random sampling and sampling variability
15 Reproducible randomness via seeding
16 The t test
17 Derivatives intuition and polynomials
18 Derivatives find minima
19 Derivatives product and chain rules

1 Overview of gradient descent
2 What about local minima
3 Gradient descent in 1D
4 CodeChallenge unfortunate starting value
5 Gradient descent in 2D
6 CodeChallenge 2D gradient ascent
7 Parametric experiments on gd
8 CodeChallenge fixed vs dynamic learning rate
9 Vanishing and exploding gradients
10 Tangent Notebook revision history

1 The perceptron and ANN architecture
2 A geometric view of ANNs
3 ANN math part 1 forward prop
4 ANN math part 2 errors loss cost
5 ANN math part 3 backprop
6 ANN for regression
7 CodeChallenge manipulate regression slopes
8 ANN for classifying qwerties
9 Learning rates comparison
10 Multilayer ANN
11 Linear solutions to linear problems
12 Why multilayer linear models dont exist
13 Multi output ANN iris dataset
14 CodeChallenge more qwerties
15 Comparing the number of hidden units
16 Depth vs breadth number of parameters
17 Defining models using sequential vs class
18 Model depth vs breadth
19 CodeChallenge convert sequential to class
20. Diversity of ANN visual representations
21 Reflection Are DL models understandable yet

1 What is overfitting and is it as bad as they say
2 Cross validation
3 Generalization
4 Cross validation manual separation
5 Cross validation scikitlearn
6 Cross validation DataLoader
7 Splitting data into train devset test
8 Cross validation on regression

1 Regularization Concept and methods
2 train and eval modes
3 Dropout regularization
4 Dropout regularization in practice
5 Dropout example 2
6 Weight regularization L1L2 math
7 L2 regularization in practice
8 L1 regularization in practice
9 Training in mini batches
10 Batch training in action
11 The importance of equal batch sizes
12 CodeChallenge Effects of mini batch size

1 What are metaparameters
2 The wine quality dataset
3 CodeChallenge Minibatch size in the wine dataset
4 Data normalization
5 The importance of data normalization
6 Batch normalization
7 Batch normalization in practice
8 CodeChallenge Batch normalize the qwerties
9 Activation functions
10 Activation functions in PyTorch
11 Activation functions comparison
12 CodeChallenge Compare relu variants
13 CodeChallenge Predict sugar
14 Loss functions
15 Loss functions in PyTorch
16 More practice with multioutput ANNs
17 Optimizers minibatch momentum
18 SGD with momentum
19 Optimizers RMSprop Adam
20 Optimizers comparison
21 CodeChallenge Optimizers and something
22 CodeChallenge Adam with L2 regularization
23 Learning rate decay
24 How to pick the right metaparameters

1 What are fully connected and feedforward networks
2 The MNIST dataset
3 FFN to classify digits
4 CodeChallenge Binarized MNIST images
5 CodeChallenge Data normalization
6 Distributions of weights pre and post learning
7 CodeChallenge MNIST and breadth vs depth
8 CodeChallenge Optimizers and MNIST
9 Scrambled MNIST
10 Shifted MNIST
11 CodeChallenge The mystery of the missing 7
12 Universal approximation theorem

1 Anatomy of a torch dataset and dataloader
2 Data size and network size
3 CodeChallenge unbalanced data
4 What to do about unbalanced designs
5 Data oversampling in MNIST
6 Data noise augmentation with devsettest
7 Data feature augmentation
8 Getting data into colab
9 Save and load trained models
10 Save the best performing model
11 Where to find online datasets

1 Two perspectives of the world
2 Accuracy precision recall F1
3 APRF in code
4 APRF example 1 wine quality
5 APRF example 2 MNIST
6 CodeChallenge MNIST with unequal groups
7 Computation time
8 Better performance in test than train

1 Project 1 A gratuitously complex adding machine
2 Project 1 My solution
3 Project 2 Predicting heart disease
4 Project 2 My solution
5 Project 3 FFN for missing data interpolation
6 Project 3 My solution

1 Explanation of weight matrix sizes
2 A surprising demo of weight initializations
3 Theory Why and how to initialize weights
4 CodeChallenge Weight variance inits
5 Xavier and Kaiming initializations
6 CodeChallenge Xavier vs Kaiming
7 CodeChallenge Identically random weights
8 Freezing weights during learning
9 Learning related changes in weights
10 Use default inits or apply your own

1 What are autoencoders and what do they do
2 Denoising MNIST
3 CodeChallenge How many units
4 AEs for occlusion
5 The latent code of MNIST
6 Autoencoder with tied weights

1 What is a GPU and why use it
2 Implementation
3 CodeChallenge Run an experiment on the GPU

12 Creating and using custom DataLoaders
1 Convolution concepts
2 Feature maps and convolution kernels
3 Convolution in code
4 Convolution parameters stride padding
5 The Conv2 class in PyTorch
6 CodeChallenge Choose the parameters
7 Transpose convolution
8 Maxmean pooling
9 Pooling in PyTorch
10 To pool or to stride
11 Image transforms

1 The canonical CNN architecture
2 CNN to classify MNIST digits
3 CNN on shifted MNIST
4 Classify Gaussian blurs
5 Examine feature map activations
6 CodeChallenge Softcode internal parameters
7 CodeChallenge How wide the FC
8 Do autoencoders clean Gaussians
9 CodeChallenge AEs and occluded Gaussians
10 CodeChallenge Custom loss functions
11 Discover the Gaussian parameters
12 The EMNIST dataset letter recognition
13 Dropout in CNNs
14 CodeChallenge How low can you go
15 CodeChallenge Varying number of channels
16 So many possibilities How to create a CNN

1 Project 1 Import and classify CIFAR10
2 Project 1 My solution
3 Project 2 CIFAR autoencoder
4 Project 3 FMNIST
5 Project 4 Psychometric functions in CNNs

1 Transfer learning What why and when
2 Transfer learning MNIST FMNIST
3 CodeChallenge letters to numbers
4 Famous CNN architectures
5 Transfer learning with ResNet 18
6 CodeChallenge VGG 16
7 Pretraining with autoencoders
8 CIFAR10 with autoencoder pretrained model

1 What is style transfer and how does it work
2 The Gram matrix feature activation covariance
3 The style transfer algorithm
4 Transferring the screaming bathtub
5 CodeChallenge Style transfer with AlexNet

1 GAN What why and how
2 Linear GAN with MNIST
3 CodeChallenge Linear GAN with FMNIST
4 CNN GAN with Gaussians
5 CodeChallenge Gaussians with fewer layers
6 CNN GAN with FMNIST
7 CodeChallenge CNN GAN with CIFAR

1 Leveraging sequences in deep learning
2 How RNNs work
3 The RNN class in PyTorch
4 Predicting alternating sequences
5 CodeChallenge sine wave extrapolation
6 More on RNNs Hidden states embeddings
7 GRU and LSTM
8 The LSTM and GRU classes
9 Lorem ipsum

1 Will AI save us or destroy us
2 Example case studies
3 Some other possible ethical scenarios
4 Will deep learning take our jobs
5 Accountability and making ethical AI

1 How to learn topic _X_ in deep learning
2 How to read academic DL papers

1 How to learn from the Python tutorial
2 Variables
3 Math and printing
4 Lists 1 of 2
5 Lists 2 of 2
6 Tuples
7 Booleans
8 Dictionaries

1 Indexing
2 Slicing

1 Inputs and outputs
2 Python libraries numpy
3 Python libraries pandas
4 Getting help on functions
5 Creating functions
6 Global and local variable scopes
7 Copies and referents of variables
8 Classes and object oriented programming

1 If else statements
2 If else statements part 2
3 For loops
4 Enumerate and zip
5 Continue
6 Initializing variables
7 Single line loops list comprehension
8 while loops
9 Broadcasting in numpy
10 Function error checking and handling

1 Printing and string interpolation
2 Plotting dots and lines
3 Subplot geometry
4 Making the graphs look nicer
5 Seaborn
6 Images
7 Export plots in low and high resolution

1. Bonus content

Udemy - A deep understanding of deep learning (with Python intro) 2025-1

About the course

Course content

Instructors

Enrolment options

Udemy - A deep understanding of deep learning (with Python intro) 2025-1

This course includes

Share this course