AI FOUNDATIONS
Neural Networks & Deep Learning
Build a strong mental model of neural networks, deep learning, backpropagation, optimization, embeddings, attention and Transformers — the foundations behind modern AI and LLMs.
Neural Networks & Deep Learning Path
Learn the concepts in order. Each module builds toward understanding Transformers, LLMs and modern AI systems.
What Is Deep Learning?
Understand what Deep Learning actually is, how it differs from traditional Machine Learning, why neural networks became important, and where Deep Learning sits on the path toward Transformers and LLMs.
Neural Network Anatomy
Build a complete mental model of a neural network — neuron, weight, bias, weighted sum, activation, layer, and parameters — through a fully hand-worked numeric example and a from-scratch implementation.
Perceptron and Decision Boundaries
Understand the perceptron, linear decision boundaries, and the XOR problem — and exactly why this limitation motivates multi-layer networks with nonlinear activations.
Activation Functions
Understand why activation functions are structurally necessary, with sigmoid, tanh, ReLU, Leaky ReLU, and softmax explained through intuition, formula, and numeric example — plus a direct proof of why linear layers alone can't work.
Forward Propagation
Trace one input all the way through a tiny neural network — input, linear transformation, activation, layer by layer, to a final prediction — hand-calculated and then verified in NumPy.
Loss Functions
Understand how a neural network measures how wrong its prediction was — MSE, MAE, binary and categorical cross-entropy — with a worked numeric example, connecting directly to LLM training.
Backpropagation
Understand backpropagation from first principles — the chain rule, computational graph, and error flowing backward through a network — with a manually computed and numerically-verified example.
Gradient Descent and the Training Loop
Assemble forward propagation, loss, backpropagation, and parameter updates into the complete training loop — batch, stochastic, and mini-batch gradient descent, epochs, iterations, and batches, all in one working example.
Optimizers and Learning Rate
Understand why plain SGD can be slow, and how Momentum, Adam, and AdamW improve on it — with a real, code-verified comparison, connecting directly to LLM training and fine-tuning.
Training Deep Networks
Understand the practical problems that show up specifically when networks get deep — vanishing and exploding gradients, weight initialization, and gradient clipping — with a real, code-verified demonstration.
Overfitting, Regularization and Normalization
Understand overfitting, dropout, weight decay, and early stopping, and precisely distinguish Batch Normalization from Layer Normalization — including why LayerNorm became especially important in Transformers.
Embeddings and Representation Learning
Understand learned representations, dense embeddings, and how they differ from hidden states and activations — with a verified cosine similarity example and the classic embedding-arithmetic demonstration, connecting directly to RAG and vector databases.
CNNs and Computer Vision
Understand why fully connected networks struggle with images, and how convolution, filters, feature maps, and pooling let networks learn hierarchical visual features directly from pixels — with a verified edge-detection example.
RNNs, LSTM and GRU
Understand why sequence data is different, how RNNs process it with a recurrent hidden state, why vanishing gradients hit them especially hard, and how LSTM/GRU gates helped — and why these limitations directly motivate attention.
Attention
Understand attention from first principles — query, key, value, similarity scores, scaling, softmax, and the weighted sum — with a complete, verified numeric computation that bridges into Transformers and LLMs.
Transformers
Understand the Transformer architecture deeply enough to remove the black box — multi-head attention, residual connections, layer normalization, positional encoding, and why GPT-style LLMs are decoder-only — with verified numeric examples.
How LLMs Actually Use Deep Learning
Trace the complete path from raw text to a generated next token — tokenization, embeddings, Transformer blocks, logits, and probabilities — integrating everything from this course, and precisely distinguishing LLM training from inference.
Deep Learning in Modern AI Engineering
The final integration module — where every Deep Learning concept from this course lives inside a complete AI/Agentic AI system architecture, plus a clear MUST KNOW / SHOULD KNOW / GOOD TO KNOW breakdown, and the bridge into the Transformers and LLM course.
WHY NEURAL NETWORKS & DEEP LEARNING?
Learn Deep Learning through the lens of modern AI
Neural networks form the foundation of modern AI. Understanding neurons, representations, loss, backpropagation, optimization, embeddings, attention and Transformers makes it much easier to understand LLMs, RAG, Generative AI and Agentic AI systems.