Learning goal: log p(x) = ELBO(q,θ) + KL(q ‖ p(z|x)). The gap is KL, and KL is nonnegative. EM alternates two operations: the E-step sets q to the true posterior, making the bound tight; the M-step raises that tight bound by changing the model. With an exact E-step and a nondecreasing M-step objective, model log-likelihood cannot decrease.

Data rug and current two-component mixture; dashed: target density.

Top: log p(x) and ELBO after each substep. Bottom: posterior-KL gap on a log scale; exact E-steps close it.

Current substep = — log p(x) / observation = — ELBO / observation = — Gap = KL(q‖p(z|x)) = — Gap after E-step = — Change in log p(x) this substep = —

Observe: advance one substep. The E-step leaves log p(x) fixed while raising ELBO to meet it; the displayed gap becomes numerical zero. It changes inference, not the generative parameters. The M-step raises the bound and likelihood; a posterior gap can reopen because θ changes and the old q need not remain the posterior. Increasing the initial q offset opens a larger gap that an exact E-step still closes. Reducing component separation can change convergence speed and the attained sample likelihood.