Machine Learning Foundations

1. L0 Learning as Fitting 0 / 4 The three ingredients of supervised learning, how the loss decides what you learn, population vs. empirical risk, and gradient descent on one page.

   1. L0.0 How an Agent Prices a House: Learning Rules from Examples
   2. L0.1 One Degree or Ten: A Loss Function Declares What Matters
   3. L0.2 A Property Agent Moves Cities: Risk and Empirical Risk
   4. L0.3 Downhill in Fog: Direction, Step Size, and Gradient Descent

   Tools this unit uses

   M0.0 Gradients and Direction M0.2 Taylor Expansion and Straight-Line Prediction M1.1 Conditional Expectation M1.2 Which Temperature Should We Forecast? MSE Selects the Conditional Mean M3.4 Strong and Weak Convergence M5.1 KL Divergence and Cross-Entropy
2. L1 Generalization: Fitting and Data Splits 0 / 5 Memorizing vs. learning, the bias–variance decomposition, the roles of train / validation / test and how leakage happens, cross-validation, and reading learning curves as a diagnosis.

   1. L1.0 Memorizing Old Exams: Overfitting and Underfitting
   2. L1.1 Ten Agents, Different Houses: Bias and Variance
   3. L1.2 Teaching and Testing with the Same Exam: Why Split Data?
   4. L1.3 Only Thirty Examples: Cross-Validation
   5. L1.4 More Data, a Different Model, or More Training? Learning Curves

   Tools this unit uses

   M1.0 Linear Combinations of Gaussians M1.1 Conditional Expectation M1.2 Which Temperature Should We Forecast? MSE Selects the Conditional Mean M5.0 Convexity and Jensen's Inequality
3. L2 Optimization in Practice 0 / 5 Minibatch gradient noise, learning rates and Adam, moving targets and EMA, regularization and early stopping, loss landscapes and initialization.

   1. L2.0 Polling a Small Sample: Minibatch Gradients and Noise
   2. L2.1 One Learning Rate Is Not Enough: Momentum and Adam
   3. L2.2 When the Model Produces Its Own Targets
   4. L2.3 Limit the Notes or Stop Practicing: Regularization and Early Stopping
   5. L2.4 One Map, Different Starting Points: Loss Landscapes and Initialization

   Tools this unit uses

   M0.0 Gradients and Direction M0.2 Taylor Expansion and Straight-Line Prediction M1.0 Linear Combinations of Gaussians M1.1 Conditional Expectation M3.0 Where Is the Walker After an Hour? From Random Walks to Brownian Motion M3.1 Stochastic Differential Equations M5.2 EMA, Moving Targets, and Stop-Gradient
4. L3 Experiments and Hyperparameter Search 0 / 4 Parameters vs. hyperparameters, grid / random / Bayesian search, nested validation and the winner's curse, seed variance and honest reporting.

   1. L3.0 Recipe or Cooking Adjustment? Parameters and Hyperparameters
   2. L3.1 One Hundred Trials: A Grid or Random Points?
   3. L3.2 Pick the Best Student: Selection Bias and Nested Validation
   4. L3.3 Three Runs, Three Answers: What Should We Report?

   Tools this unit uses

   M1.0 Linear Combinations of Gaussians M5.0 Convexity and Jensen's Inequality
5. L4 Architectures and Inductive Biases 0 / 6 MLPs and universal approximation, backpropagation, convolution and translation equivariance, U-Nets and residuals, attention and Transformers, and how conditioning and time are injected.

   1. L4.0 If MLPs Can Approximate Everything, Why Use Other Architectures?
   2. L4.1 A Defect on the Production Line: Backpropagation and Sensitivity
   3. L4.2 The Same Pattern in Another Position: Convolution and Equivariance
   4. L4.3 How can one kernel handle texture, objects, and an entire scene?
   5. L4.4 Can content decide where to look?
   6. L4.5 Where should instructions enter a shared model?

   Tools this unit uses

   M0.0 Gradients and Direction M0.2 Taylor Expansion and Straight-Line Prediction M0.3 Total Derivatives, Integral Differentiation, and JVPs M1.1 Conditional Expectation
6. L5 Probabilistic Modeling and Generative Evaluation 0 / 4 Maximum likelihood and KL, latent variables and the ELBO, autoregressive factorization and independence assumptions, and how to evaluate a generative model.

   1. L5.0 How should we score how well a model explains the data?
   2. L5.1 What can we optimize when the required marginal is intractable?
   3. L5.2 Can a high-dimensional probability become a sequence of one-dimensional probabilities?
   4. L5.3 Which number tells us whether a generative model is good?

   Tools this unit uses

   M1.0 Linear Combinations of Gaussians M1.3 Bayes' Rule and the Posterior M4.0 Markov Chains and Transition Matrices M5.0 Convexity and Jensen's Inequality M5.1 KL Divergence and Cross-Entropy M6.1 How Far Apart Are Two Population Maps? The W₂ Distance

Lit notes are what this browser has read; the record stays on this device. Clear