Learning goal: Empirical risk is computable from observations; population risk is the expected loss of interest. In this polynomial example, increasing degree improves the ideal training fit but can worsen population risk. Sample size, noise, model family, and numerical fitting affect the curves.

Points: training observations; dashed gray: sin(2πx); solid: current polynomial least-squares fit.

Log vertical axis: training MSE, population risk, irreducible error σ², and the current degree marker.

Observe: ideal unregularized minimum training error cannot rise for nested polynomial spaces. Computed fits can deviate because of numerical stabilization. Population risk need not decrease. More data can reduce a generalization gap; setting σ=0 does not guarantee correct extrapolation or identify a function from finitely many observations. Noise is not required for finite-data generalization failure.