Learning goal: with self-generated targets, a constant function can make consistency loss zero. This eight-feature linear Fourier toy separates three interventions: an anchor specifies a nontrivial target; EMA changes target dynamics; stop-gradient changes the update field without by itself specifying a useful solution or guaranteeing stability.

Dashed gray: sin(2πx), peak-to-peak range 2.00; solid: trained predictor; dots: anchors.

Log consistency loss (blue) and output range (red) versus steps; horizontal: target range 2.00.

Observe: without anchors, the loss reaches about 4.9e-14 while the output range falls to about 4.9e-6 here. Stop-gradient and EMA change the dynamics but do not eliminate the constant solution in this toy: neither alone guarantees noncollapse. Adding enough informative anchors restores a nontrivial fit in this experiment. The note's nonlinear example can instead diverge with a different update field. Inspect representation behavior as well as loss. )