Learning goal: for nested families and exact empirical minimization, training error cannot increase with capacity. This alone does not establish generalization. This example's population risk is U-shaped, with an optimum that changes with n. Such a curve is an observation under this setup, not a universal law.

Points: training data; dashed gray: truth; solid: current degree-d fit.

Log training error and population risk versus degree; horizontal: σ²; triangle: best tested degree; vertical: current d.

n = 30 Best tested degree d = — Minimum tested risk = — Current training error = — Current population risk = —

Observe: increasing n can shift and flatten the optimum in this example. At σ=0, finite-data fitting can still generalize poorly, and the high-degree risk can still rise. The repeated-run bands show that risk estimates themselves vary. The next note decomposes the expected squared error more precisely.