Every order has a chain rule; restricted models still differ

Learning goal: exact factorization in every order does not guarantee identical estimates under finite data and restricted conditionals. Compare context and ordering through held-out NLL and sequential generation rounds.

Test NLL across orders. Conditional tables are fitted on training data; the true model's score on this finite test set is a reference, not a universal sample-score lower bound.

Views: ordering at different context sizes, and sequential generation rounds versus one-pass generation.

Variable count d Conditional context Dependence strength

Training samples n

True-model test NLL, reference = — Best tested order = — Worst tested order = — Worst − best = — Full-prefix worst − best = — Autoregressive sampling rounds = —

Observe: compare order-to-order variation for full and truncated context, then vary dependence and sample size. Truncation can hide different dependencies; full-prefix estimates still vary with finite samples and smoothing. True entropy lower-bounds population cross-entropy, not every score on a finite test sample. The cost plot counts generation rounds, not wall-clock time.