Explore completed-output selection, intermediate SMC, and source MCMC/SPT in your chosen order. Sample, trace ancestry and chains, then build the final comparison.
Prefer continuous reading? Skill manual Read the note (Chinese) · U7.3 →
The user still wants varied dog photos and prefers certain hats, lighting, and compositions. Their scoring program accepts a finished image and returns a number, but cannot say which pixel to change or how. Keep the generative parameters fixed. Can scores alone make preferred results appear more often?
Write a finished image as and its reward as . A scorer that we can query without obtaining usable derivatives is a black-box reward. External simulators, collision checks, and human evaluators can provide such signals. The reward need not be a probability or differentiable.
The score 7 is hypothetical, not a probability or model measurement. Arrows only mean submitting an image and receiving its score. The program provides no direction for revision. What could we do with this number?
Select the image to enlarge“If we do not know how to revise, generate a few images as before. After the evaluator scores them, select one to return. At least we do not need revision directions.”
Mugi keeps each image’s generation unchanged and applies the preference through selection after completion. We could choose the best, or make high scores more likely to be sampled. Do these rules produce the same proportions?
“Finishing every image before scoring might be expensive. If we could guess a draft’s future, could we keep more copies of promising drafts and spend less time on the others?”
Lumi wants to change which drafts receive the remaining generation computation. The evaluator still understands only finished images, so intermediate judgments must estimate the future rather than treat noise as a finished photo. Estimates can be wrong; discarding every alternative immediately is risky.
“If the same start always generates the same image, copying a draft only repeats an answer. Could we try different starts and find ones that tend to draw results we like?”
Nori preserves the generation rule and changes which starting points we use. Every new start still requires generation and scoring. Finding a high-scoring start after a few trials does not yet mean sampling a specified distribution.
Picking the best is reasonable if we want one high-scoring photo. But the user also wants alternatives. Our question is whether repeated outputs approach this reward tilt:
Here is the base distribution, controls reward influence, and averages the reward multiplier over base-generated finished images. Assume ; dividing by it restores total mass to . Discrete results use probabilities and continuous images use densities. An exact point in the latter need not have a nonzero probability of being sampled.
This target raises high-reward relative weights while retaining base proportions. U7.2 derived its motivation and optimality. Today we ask how to sample, without updating the model. First assume the base sampler exactly produces ; we will return to actual model error later.
Reuse one executable base: . is a completed datum and a noisy state. Numbers on the axis are toy data, not live dog photographs. Every route actually samples this distribution and compares it with a numerical reward-tilt reference.
Choose the order. Each route includes prediction, actual sampling plots, full derivations, and limitations. Only returning from its final step marks it fully explored. You may return early and resume later.
Select completed outputs: Best-of-N and IW
“Generate images and pick the highest score. We can already do that, right?”
Now that Mugi has proposed generating more images, consider this scene. The left side contains separate candidates; only one is returned on the right. Every candidate must be finished and scored.
Select the image to enlargeIndependently generate from the base. Score them, find the highest-scoring index , and return that image:
This is Best-of-. The selects an index; it updates neither pixels nor parameters. Ties can use a fixed or random rule. No reward gradient is needed, and each candidate’s original sampling path is unchanged.
Generate a fresh batch of , select one, and independently repeat the whole run times to collect outputs. is the candidates compared per run; is the number of collected outputs. Increasing reveals this rule’s proportions, which still need to be checked against .
Actually draw each batch from the mixture and select by reward. Observe rank versus multiplier before calculating the discrete example.
Predict, then inspect the distribution
Increasing N can raise reward. Does that make the output match the dotted reward tilt? What if only λ changes while rankings stay fixed?
A uses N = 1; B uses your N. Each of 1,000 independent batches generates fresh candidates and returns the highest-reward one.
The gray curve shows the original mixture. Running adds actual samples; a reference curve does not establish that it was sampled.
Explain using the plot and numbers
Best-of-N selects by rank. Changing λ without changing ranks leaves winners unchanged, while the tilt changes. More output draws reveal the rule but do not change it into the target. The candidate table records actual draws and selections.
This explains the standard comparison. After changing controls or Python, use the settings and plots you actually ran.
Every candidate is drawn from the same Gaussian mixture. The histogram contains 1,000 independent returns. The dotted curve is a numerical tilt reference, not the theoretical Best-of-N density.
Change the parameters or the sampling rule. Python loads when you press Run and executes in your browser. Each run starts fresh; your edits are not saved.
The first opening imports the controls above; reopening preserves your edits. Settings are imported, not accumulated trained parameters or samples, so finite-sample plots need not match exactly.
The code implements this method and plots its distribution with Matplotlib. The first opening imports current controls; every run samples afresh. Change one parameter at the top. The native plot is not a record of edited Python output.
import math
import numpy as np
import matplotlib.pyplot as plt
# 1. SETTINGS -- change one, run again, and explain the plotted evidence.
strength = 2
N = 8 # candidates per returned sample
reward_kind = "right" # "right" or "two"
rng = np.random.default_rng(42)
sigma = 0.55
grid = np.linspace(-6, 6, 481)
dx = grid[1] - grid[0]
# 2. FIXED MODEL -- analytic functions, no network training in this cell.
def pdf(x, mu=0.0, var=1.0):
return np.exp(-0.5 * (x-mu)**2 / var) / np.sqrt(2*np.pi*var)
def R(x):
return x if reward_kind == "right" else -(np.abs(x)-2.5)**2
def moments(t):
a = np.exp(-8*t)
return np.sqrt(a), a*sigma**2 + 1-a
def sample_base(size, t=0.0):
signal, var = moments(t)
mu = np.where(rng.random(size) < 0.5, -2.0, 2.0)
return signal*mu + np.sqrt(var)*rng.normal(size=size)
base = 0.5*pdf(grid, -2, sigma**2)+0.5*pdf(grid, 2, sigma**2)
target = base*np.exp(strength*R(grid))
target /= target.sum()*dx # finite-grid quadrature reference
reference = target
reference_label = "Tilt reference (quadrature on [-6,6])"
extra_figure = None
# 3. GENERATION -- 1000 independent batches; return ONE sample per batch.
batch = sample_base((1000, N))
rewards = R(batch)
selected = np.argmax(rewards, axis=1)
samples = batch[np.arange(1000), selected]
print("Generated candidates and reward calls:", 1000*N)
print("First batch:", batch[0], "selected index:", selected[0])
print("Finite N does not guarantee the reference tilt.")
# 4. PLOT AND CHECK -- the histogram is from the samples above.
fig, ax = plt.subplots(figsize=(8, 3.6))
ax.plot(grid, base, "--", color="gray", label="Original base")
ax.plot(grid, reference, ":", color="black", label=reference_label)
ax.hist(samples, bins=np.linspace(-6,6,49), weights=np.ones(len(samples))/(len(samples)*0.25), alpha=0.5, label="Actual generated samples")
ax.set(xlabel="clean x0", ylabel="density", xlim=(-6,6))
ax.legend(fontsize=8)
fig.tight_layout()
print("Outputs:", len(samples), "mean:", np.mean(samples), "right share:", np.mean(samples > 0))
print("Outside the plotted window:", np.sum(np.abs(samples)>6))
plt.show()
Simplify results to , each with base probability . Applying to the scores gives multipliers . These are multipliers, not raw rewards, and has the highest reward. The desired target probabilities are .
Generate only two candidates each run: Best-of-. What is the probability of returning ? Compute the probability that neither is , then take the complement. Does the result equal the target’s ?
You can also think first, then read the explanation. Your writing stays on this page and disappears when you reload.
The candidates are independent, and wins whenever present. Thus
Returning requires two copies of . Returning requires no but at least one . These probabilities sum to but do not have the desired ratio . Best-of- depends on beating other candidates; reward tilting depends on base weights times multipliers.
Even changing multiplier to leaves Best-of- unchanged if the ranking stays the same. The target ratio would become . A ranking-only rule cannot generally preserve multiplier information.
Keep Best-of-, generate fresh batches every day, and collect one hundred thousand outputs. Which quantity will the frequency of approach?
Each run still returns C with probability five ninths. More collected outputs do not change that rule.
Seven tenths is the specified target. Repeating a different output distribution does not turn it into that target.
Each run still compares only two candidates, sometimes without C. We increased repetitions, not candidates per run.
“If the table happens to contain one , one , and one , could we select them with probabilities ? would remain common while the others still have a chance.”
This samples by weight conditional on a fixed batch. Real base-generated batches need not contain one of each. Where are the base proportions already represented, and which multiplier should each candidate receive?
Common base results are already more likely to enter the batch. To represent , we need the target’s relative increase over the sampling source. Where the base is positive, simplify
What cancels? What is shared by every candidate? If we will divide by the batch’s total weight, must that shared constant be computed first?
You can also think first, then read the explanation. Your writing stays on this page and disappears when you reload.
The base cancels, leaving . The shared cancels when normalizing, so candidate only needs positive weight
Here is a relative multiplier; is the probability of selecting candidate given this batch. Candidate frequencies already represent the base; selection supplies the reward multiplier. We do not multiply by base probability again.
Correcting contributions by the target-to-source ratio is the core of importance sampling. Here we perform importance weighting (IW), then sample an index with normalized weights and return its finished image.
Let be a quantity to average, such as a class indicator, with finite absolute expectation under the target. Multiply and divide the expectation’s integrand by the base density:
Both expectations average base-generated images. The numerator also multiplies by , and the denominator is . Estimate both means with the same independent candidates; their factors cancel:
This is self-normalized importance sampling. Compute a weighted average directly, without selecting another image. Finite-sample ratios are generally biased. With independent candidates and the relevant integrability, increasing makes both sample means converge to their expectations, and their ratio to the target average.
Replace argmax with sampling a normalized-weight index in each fresh batch. Compare N = 1 and larger N and inspect the target approximation.
Predict, then inspect the distribution
Candidates already come from the base. How does selecting by exponential multipliers differ between N = 1 and a larger N?
A uses N = 1; B uses your N. Draw fresh candidates in every batch, normalize their multipliers, and sample one index.
The gray curve shows the original mixture. Running adds actual samples; a reference curve does not establish that it was sampled.
Explain using the plot and numbers
At N = 1 the only normalized weight is 1, so output remains the base. More candidates improve coverage, but finite self-normalized resampling is still approximate. Conditional selection within one batch differs from the output law across batches.
This explains the standard comparison. After changing controls or Python, use the settings and plots you actually ran.
Base frequencies are already represented by candidate generation. Weights supply exp(λR), without multiplying the base density again. Each of 1,000 batches returns one actual sampled candidate.
Change the parameters or the sampling rule. Python loads when you press Run and executes in your browser. Each run starts fresh; your edits are not saved.
The first opening imports the controls above; reopening preserves your edits. Settings are imported, not accumulated trained parameters or samples, so finite-sample plots need not match exactly.
The code implements this method and plots its distribution with Matplotlib. The first opening imports current controls; every run samples afresh. Change one parameter at the top. The native plot is not a record of edited Python output.
import math
import numpy as np
import matplotlib.pyplot as plt
# 1. SETTINGS -- change one, run again, and explain the plotted evidence.
strength = 2
N = 8 # candidates per returned sample
reward_kind = "right" # "right" or "two"
rng = np.random.default_rng(42)
sigma = 0.55
grid = np.linspace(-6, 6, 481)
dx = grid[1] - grid[0]
# 2. FIXED MODEL -- analytic functions, no network training in this cell.
def pdf(x, mu=0.0, var=1.0):
return np.exp(-0.5 * (x-mu)**2 / var) / np.sqrt(2*np.pi*var)
def R(x):
return x if reward_kind == "right" else -(np.abs(x)-2.5)**2
def moments(t):
a = np.exp(-8*t)
return np.sqrt(a), a*sigma**2 + 1-a
def sample_base(size, t=0.0):
signal, var = moments(t)
mu = np.where(rng.random(size) < 0.5, -2.0, 2.0)
return signal*mu + np.sqrt(var)*rng.normal(size=size)
base = 0.5*pdf(grid, -2, sigma**2)+0.5*pdf(grid, 2, sigma**2)
target = base*np.exp(strength*R(grid))
target /= target.sum()*dx # finite-grid quadrature reference
reference = target
reference_label = "Tilt reference (quadrature on [-6,6])"
extra_figure = None
# 3. GENERATION -- 1000 independent batches; return ONE sample per batch.
batch = sample_base((1000, N))
rewards = R(batch)
weights = np.exp(strength*(rewards-rewards.max(axis=1, keepdims=True)))
weights /= weights.sum(axis=1, keepdims=True)
selected = np.sum(rng.random((1000, 1)) > np.cumsum(weights, axis=1), axis=1).clip(0, N-1)
samples = batch[np.arange(1000), selected]
print("Generated candidates and reward calls:", 1000*N)
print("First batch:", batch[0], "selected index:", selected[0])
print("Finite N does not guarantee the reference tilt.")
# 4. PLOT AND CHECK -- the histogram is from the samples above.
fig, ax = plt.subplots(figsize=(8, 3.6))
ax.plot(grid, base, "--", color="gray", label="Original base")
ax.plot(grid, reference, ":", color="black", label=reference_label)
ax.hist(samples, bins=np.linspace(-6,6,49), weights=np.ones(len(samples))/(len(samples)*0.25), alpha=0.5, label="Actual generated samples")
ax.set(xlabel="clean x0", ylabel="density", xlim=(-6,6))
ax.legend(fontsize=8)
fig.tight_layout()
print("Outputs:", len(samples), "mean:", np.mean(samples), "right share:", np.mean(samples > 0))
print("Outside the plotted window:", np.sum(np.abs(samples)>6))
plt.show()This run produced , with multipliers . Given this batch, what is the probability of returning ?
Normalization uses this batch’s total weight. There is no B in this run to add to the denominator.
Weighted sampling can still select A. Only maximization would always select C from this batch.
is the proportion of in the whole target. Our table contains only , with total weight , and no weight from . Selection given this batch can only choose among its members.
The extreme case is : regardless of weight, there is only one candidate, so the output remains the base. Weighted sampling approximates the target with finite candidates; it does not exactly reproduce in every batch.
Each run now independently generates base images, scores them, computes multipliers, normalizes, and samples one. More repetitions reveal the fixed- output law. More candidates can improve base coverage and the finite-candidate approximation.
But every candidate must still be completed and scored. If each matches the request with probability , the chance of at least one hit among independent candidates is . For and , it is only about . This concerns a hit somewhere in the batch, not the probability that the weighted selection succeeds.
Mugi has put scores into selection, but Lumi’s cost question remains: could later computation go to another draft when one already looks unpromising halfway through?
You implemented Best-of-N and IW and read candidate weights, finite-N output laws, repeated runs, and generation cost. Return to explore intermediate computation or source sampling.
Resample drafts: SMC and the full FK derivation
“Ask the denoiser for a predicted finished image, then show the evaluator. Or complete the same draft independently several times and average the resulting multipliers. Could we compare drafts before finishing?”
We can try, while separating estimates from outcomes. The current noisy state is , its denoised estimate is , and only reaching the endpoint gives . In the VP representation, for ,
Here is predicted noise and is its amplitude. Subtract predicted noise and divide out the signal scale to get a clean estimate. This is not an already determined future image.
Estimating future weight with only queries reward, without differentiation. With multiple independent completions, compute each first, then average. Those trial completions also cost generation. Both are intermediate estimates, not guaranteed exact future averages.
Lumi shows a denoised estimate to the evaluator and receives a high score. How can we use that number now?
The scored input is a predicted completion. This avoids gradients but retains uncertainty about the future.
The scored input is an estimate, not the image actually reached at the endpoint. They may differ greatly at high noise.
Returning a score does not imply a derivative with respect to the noisy image. We are comparing candidates.
Suppose the positive incremental weights for three drafts are , normalizing to . We want three candidates for the next segment, drawing a draft with these probabilities each time, allowing repeated selections.
Across three draws, how many copies does the third draft receive on average? Must every run contain exactly that number? Separate expected copies from actual copies.
You can also think first, then read the explanation. Your writing stays on this page and disappears when you reload.
Each draw selects the third draft with probability , giving expected count . The first two have expected counts , but actual counts are integers. One run might draw or three copies of . These are possible outcomes, not prescribed copying lists.
Drawing candidates again by probability is resampling. Afterward, each copy receives equal weight . Higher-weight drafts have more descendants on average and receive more future computation; the total remains three. Neither model nor scorer is retrained.
For now, were assumed incremental multipliers for this segment. We still need to derive how future estimates give those incremental weights.
Call an in-progress candidate a particle and keep simultaneously. Diffusion generates from high noise toward . Choose checkpoints , where counts segments and is draft at time .
From to , first move each draft with the original reverse sampler: propagate. Compute its incremental weight : weight. Normalize over the batch to get , then draw indices with those probabilities: resample.
Repeating “propagate → weight → resample” is the basic operation of Sequential Monte Carlo (SMC). Here every checkpoint resamples, resetting weights to . At , uniformly select one of the final images to return. We change which candidates have descendants, not .
Each particle has a position and ancestor. Step through one population, then draw one return from each of 300 independent populations. Deliberately misestimate H and inspect whether terminal reward correction can restore a lost mode. The full ratio derivation follows.
Predict, then inspect the distribution
If an intermediate estimate wrongly favors only the right, can finite particles retain the left peak after terminal correction? Which share ancestors?
A uses the posterior multiplier; B uses a deliberately right-biased intermediate estimate. M, checkpoints, terminal reward, and seeds stay fixed.
You can inspect one population first. Each click performs one action: propagate → incremental weights → resample.
The gray curve shows the original mixture. Running adds actual samples; a reference curve does not establish that it was sampled.
Explain using the plot and numbers
Each child divides by its own parent’s H; the terminal H uses the actual exp(λR), so intermediate H values telescope on ideal paths. Finite M can still lose a mode, and increasing M need not improve every seed. Colors track one population; the main histogram returns one sample from each of 300 independent populations, not independent descendants within one run.
This explains the standard comparison. After changing controls or Python, use the settings and plots you actually ran.
This toy uses exact reverse conditionals of the known mixture VP process, isolating finite-particle and estimator effects. H_K = 1 and H_0 = exp(λR). Each stage normalizes incremental weights and multinomially resamples M particles. Posterior H uses an analytic integral, without hidden rollouts.
Change the parameters or the sampling rule. Python loads when you press Run and executes in your browser. Each run starts fresh; your edits are not saved.
The first opening imports the controls above; reopening preserves your edits. Settings are imported, not accumulated trained parameters or samples, so finite-sample plots need not match exactly.
The code implements this method and plots its distribution with Matplotlib. The first opening imports current controls; every run samples afresh. Change one parameter at the top. The native plot is not a record of edited Python output.
import math
import numpy as np
import matplotlib.pyplot as plt
# 1. SETTINGS -- change one, run again, and explain the plotted evidence.
strength = 2
M = 24 # particles per population
estimator = "posterior" # "posterior", "plugin", "misleading"
checkpoints = 6
reward_kind = "two" # "right" or "two"
rng = np.random.default_rng(42)
sigma = 0.55
grid = np.linspace(-6, 6, 481)
dx = grid[1] - grid[0]
# 2. FIXED MODEL -- analytic functions, no network training in this cell.
def pdf(x, mu=0.0, var=1.0):
return np.exp(-0.5 * (x-mu)**2 / var) / np.sqrt(2*np.pi*var)
def Phi(x):
return 0.5 * (1 + np.vectorize(math.erf)(np.asarray(x) / np.sqrt(2)))
def R(x):
return x if reward_kind == "right" else -(np.abs(x)-2.5)**2
def moments(t):
a = np.exp(-8*t)
return np.sqrt(a), a*sigma**2 + 1-a
def sample_base(size, t=0.0):
signal, var = moments(t)
mu = np.where(rng.random(size) < 0.5, -2.0, 2.0)
return signal*mu + np.sqrt(var)*rng.normal(size=size)
def sigmoid(v):
return np.exp(-np.logaddexp(0, -v))
def post_params(x, t):
signal, var = moments(t)
right = sigmoid(4*signal*x/var)
gain = signal*sigma**2/var
left_mu = -2 + gain*(x + 2*signal)
right_mu = 2 + gain*(x - 2*signal)
variance = sigma**2*(1-np.exp(-8*t))/var
return right, left_mu, right_mu, variance
def posterior_mean(x, t):
w, m0, m1, v = post_params(x, t)
return (1-w)*m0+w*m1
def h(x, t):
if t == 0:
return np.exp(strength*R(x))
w, m0, m1, v = post_params(x, t)
if reward_kind == "right":
return (1-w)*np.exp(strength*m0+0.5*strength**2*v)+w*np.exp(strength*m1+0.5*strength**2*v)
d = 1+2*strength*v
def component(mu):
result = 0.0
for side in [-1, 1]:
center = (mu+2*strength*v*side*2.5)/d
factor = np.exp(-strength*(mu-side*2.5)**2/d)/np.sqrt(d)
result += factor*Phi(side*center/np.sqrt(v/d))
return result
return (1-w)*component(m0)+w*component(m1)
base = 0.5*pdf(grid, -2, sigma**2)+0.5*pdf(grid, 2, sigma**2)
target = base*np.exp(strength*R(grid))
target /= target.sum()*dx # finite-grid quadrature reference
reference = target
reference_label = "Tilt reference (quadrature on [-6,6])"
extra_figure = None
# 3. SMC -- exact reverse transitions of the known mixture; finite M resampling.
def reverse_base(x, current_t, next_t):
sig0, var0 = moments(next_t)
sig1, var1 = moments(current_t)
r = np.exp(-4*(current_t-next_t))
right = sigmoid(4*sig1*x/var1)
mu = np.where(rng.random(x.shape) < right, 2.0, -2.0)
gain = r*var0/var1
return sig0*mu+gain*(x-sig1*mu)+np.sqrt(max(0, var0-gain*gain*var1))*rng.normal(size=x.shape)
def H(x, t):
if t == 1:
return np.ones_like(x) # H_K = 1
if t == 0:
return np.exp(strength*R(x)) # actual terminal reward
if estimator == "posterior":
return np.maximum(1e-100, h(x, t))
if estimator == "plugin":
return np.maximum(1e-100, np.exp(strength*R(posterior_mean(x, t))))
return np.exp(np.clip(4*strength*x, -100, 100)) # deliberately wrong intermediate estimate
runs = 300
x = sample_base((runs, M), 1.0)
ancestor = np.tile(np.arange(M), (runs, 1))
trace = [x[0].copy()]
lineage = [ancestor[0].copy()]
for k in range(checkpoints):
t = 1-k/checkpoints
next_t = 0 if k+1 == checkpoints else 1-(k+1)/checkpoints
proposed = reverse_base(x, t, next_t)
log_weights = np.log(H(proposed, next_t))-np.log(H(x, t))
weights = np.exp(log_weights-log_weights.max(axis=1, keepdims=True))
weights /= weights.sum(axis=1, keepdims=True)
# Each proposed child is divided by its OWN parent's H before resampling.
cdf = np.cumsum(weights, axis=1)
parents = np.stack([np.searchsorted(cdf[i], rng.random(M)).clip(0, M-1) for i in range(runs)])
x = np.take_along_axis(proposed, parents, axis=1)
ancestor = np.take_along_axis(ancestor, parents, axis=1)
trace.append(x[0].copy())
lineage.append(ancestor[0].copy())
# ONE output from each independent population, not 300*M independent descendants.
samples = x[np.arange(runs), rng.integers(M, size=runs)]
print("Independent populations:", runs, "particles each:", M)
print("Reverse transitions:", runs*M*checkpoints, "H calls:", 2*runs*M*checkpoints)
print("Surviving ancestors in the first population:", len(np.unique(ancestor[0])))
extra_figure, ax = plt.subplots(figsize=(8, 3.2))
for i, (positions, origins) in enumerate(zip(trace, lineage)):
ax.scatter(positions, np.full(M, i), c=origins, cmap="tab20", s=15, vmin=0, vmax=max(1,M-1))
ax.set(xlabel="noisy state x_t (last row is clean)", ylabel="checkpoint in generation order")
ax.invert_yaxis()
extra_figure.tight_layout()
# 4. PLOT AND CHECK -- the histogram is from the samples above.
fig, ax = plt.subplots(figsize=(8, 3.6))
ax.plot(grid, base, "--", color="gray", label="Original base")
ax.plot(grid, reference, ":", color="black", label=reference_label)
ax.hist(samples, bins=np.linspace(-6,6,49), weights=np.ones(len(samples))/(len(samples)*0.25), alpha=0.5, label="Actual generated samples")
ax.set(xlabel="clean x0", ylabel="density", xlim=(-6,6))
ax.legend(fontsize=8)
fig.tight_layout()
print("Outputs:", len(samples), "mean:", np.mean(samples), "right share:", np.mean(samples > 0))
print("Outside the plotted window:", np.sum(np.abs(samples)>6))
plt.show()Mugi says, “To spend time on good drafts, why not keep only the highest-scoring one and copy it times?” What does this overlook?
Finite candidates need exploration. We score an estimated future, not a known best final path.
Parameters remain fixed. Keeping one ancestor changes candidates and computation allocation, not model training.
With fresh sampler randomness, copies of one draft may reach different results. Along the same deterministic ODE, however, identical states follow the same route. Ten copies still give one answer.
Shrinking to a few ancestors and narrowing exploration is particle degeneracy. Random normalized-weight resampling can also cause it, especially with concentrated weights. Any added exploration or perturbation must be checked for target preservation; copy count alone does not establish diversity.
Having only a few ancestors and narrowing exploration is particle degeneracy. Random resampling can still cause it when weights concentrate. Copies need later randomness to separate; under the same deterministic rule, more copies do not create different answers.
“We know how to resample. But suppose a draft’s future multiplier is first estimated as , then is at completion. Multiplying by both and gives , although we only wanted a factor of . What was counted twice?”
Let be a positive estimate at checkpoint . It may be a cheap estimate, not U7.2’s exact posterior average. Set at the start and at completion, with positive intermediate values.
Each complete base path should receive the terminal multiplier only once. We are checking the accumulated product, not how high each separate estimate is. First follow a worked example, then choose the next operation yourself.
Split this path into two segments: , , and . Open one operation at a time and see how the goal determines the tool.
From cumulative multipliers to each segment’s increase
Solve “ times what equals ?” The first multiplier is
Here are the same path’s estimates at its two checkpoints. It has already accumulated a factor of ; the next segment cannot pretend otherwise.
Solve “ times what equals ?” The increment is
Multiplying directly by would give ; the difference is not the missing factor either. We divide because the unknown is a factor in a product.
Connect the segments:
The intermediate estimate helps allocate computation, then cancels against the next ratio. Reward is not ignored; the same terminal preference accumulates only once.
Try another path: its previous checkpoint has accumulated a factor of , and the new estimate calls for . Which multiplier should this segment add? First identify the missing multiplicative factor, then choose a tool.
The new-to-old ratio supplies only what the cumulative product lacks, without recounting the previous part.
The difference 9 is additive. Multiplying the existing 3 by it gives 27 rather than the desired 12.
An existing factor of 3 times 12 accumulates to 36, counting the earlier contribution again.
Now split one path into three segments. Its positive estimates from start through two intermediate checkpoints to finish are . For now these denote values along that one path; we will restore state inputs shortly.
Each segment uses new divided by old, giving the product
Which terms cancel, and what remains? With start value and the true terminal reward multiplier, explain why intermediate estimates need not be exact to preserve the ideal full-path multiplier.
You can also think first, then read the explanation. Your writing stays on this page and disappears when you reload.
In the three-segment product, each appear once above and below the fraction bar, leaving . Restore the segment states and check the needed conditions.
From ratios along one path to the terminal multiplier
Call this segment’s multiplier function a potential:
The numerator is the estimate after the segment, and the denominator is the estimate before it. Dividing by another draft’s old estimate would destroy the matching values needed for cancellation.
Our main construction uses the original sampler for this segment, so candidate has incremental weight
Normalize by the sum of this batch’s values to obtain selection probabilities. A different proposal also requires correcting the sampling source; an extra section below expands that ratio.
Multiply segment ratios along the same ancestral path. As in the three-segment example, intermediate values cancel, leaving
This checks a product along one path, not an average across particles. Different paths have their own intermediate states and estimates.
Set and . Then
Intermediate estimates allocate computation; endpoint choices determine the accumulated multiplier. Positive estimates make the ratios defined. This construction preserves the ideal terminal tilt, but finite- drafts may have been discarded already, so actual approximation still depends on those estimates.
Each path now receives only . Yet multiple paths may reach the same finished result. What mathematical operation gives that result’s probability in the new distribution?
Every compatible path can contribute, retaining its original path probability.
Keeping only the most likely path removes contributions from other paths to the same endpoint. That selects a path rather than calculating total probability.
Base paths need not be equally likely. Equally averaging multipliers while discarding path probabilities loses the base proportions.
Try a four-path base example: two paths reach with probabilities ; two reach with probabilities . Terminal multipliers are for and for . The weighted masses are
A shared terminal multiplier can leave the sum. Equivalently, first add base path probabilities to obtain and , then multiply by . The normalizing total is , giving probabilities .
This is why we chose “accumulate all paths.” Keeping one path would omit the other’s mass.
Use the original sampler’s path distribution to connect full-path multipliers with finished-image proportions. Write the checkpoint sequence as . First define a path target , then check its endpoint distribution.
Weight paths, then accumulate intermediate possibilities
Here denotes the base probability measure on these paths. The product uses the state inputs defined above. The total weighted base-path mass is assumed finite and nonzero.
The new multiplier depends only on endpoint . Any two paths reaching the same result receive the same terminal weight, while retaining their original relative probabilities.
The four paths gave weighted masses 0.3 for A and 2.1 for B. At a fixed endpoint every compatible path receives the same terminal multiplier, so factor it out of the sum over intermediate states. Continuous spaces use an integral while retaining each base path’s contribution.
If the base sampler ends at , accumulating all paths for each endpoint, just as in the four-path example, gives
This denotes probabilities for discrete results and densities for continuous ones. We defined path weighting, then derived its terminal target. No particle count appears in the definition, and no image has yet been sampled.
This potential-weighted path framework is a Feynman–Kac (FK) model. SMC is the finite-particle sampling algorithm approximating it. The derivation specifies an ideal target; finite-draft error and degeneracy require separate checks.
Singhal and colleagues’ FK steering applies this construction to diffusion guidance: choose intermediate estimates, potentials, proposals, and resampling times, then sample with particle methods. Kim, Kim, and Park’s DAS (Diffusion Alignment as Sampling) also uses SMC, gradually increasing intermediate-target preference through reward tempering.
Relate these names to two layers: how targets and potentials are constructed, and how they are sampled. FK and SMC are not mutually exclusive methods.
Our main construction propagates with the original sampler, so proposal and base transition coincide. If another transition density proposes states while the base density is , the weight must also correct their ratio. With resampling each segment and equal previous weights,
The first factor is the target’s increase over the base path. The second compensates for drawing from the proposal rather than the base transition, for the same reason as finished-image IW. The proposal must cover regions with target mass for this ratio to work.
Only when does the ratio become , recovering . Here is an incremental weight function; below, is a deterministic generator. Their inputs and subscripts distinguish the roles.
An intermediate is poor but remains positive. We keep same-path ratios, , and the true terminal reward for . Which claim follows from the construction?
Accumulated multipliers determine the path target; actual sampling produces finite-particle error and degeneracy.
Poor estimates can eliminate drafts that would later be good. A correct product does not establish sufficient finite-M exploration.
Cancellation leaves . If the last reward still scores a predicted image instead of true , the remaining terminal weight may be different.
Likewise, a nonconstant without a matching initial correction leaves an extra . More particles can improve approximation, but cannot automatically repair a changed FK target.
Lumi’s complete procedure is now clear. Draw original-sampler starts. At each segment, retain old states, propagate, and divide each new by its own predecessor’s old . Normalize and independently resample indices with replacement. Equal-weight copies enter the next segment. At completion, uses the finished image’s true reward.
The final images define an empirical distribution:
Here puts a point mass at image . Estimate a category’s frequency by counting its members and dividing by . Equal weights do not imply independent samples, since particles may share ancestors. Uniformly selecting one returns this run’s output.
Let be a bounded image statistic, such as the indicator of . With fixed checkpoints, correct weights, and the corresponding integrability conditions, SMC consistency states
The left side averages particles in one run; the right side is the target average. “In probability” means the probability of exceeding any fixed positive error tolerance tends to . It does not require independent particles within the run.
Increasing improves each run’s FK-target approximation. Independently repeating a fixed- sampler times only clarifies its output proportions, which may remain biased. This limit also does not repair an incorrectly specified terminal weight.
No. Effective sample size (ESS) describes concentration of normalized weights:
With equal weights, the squared sum is , giving ESS . If one weight approaches , the squared sum approaches , giving ESS near . This measures weight concentration, not independent ancestor count.
We can resample only when ESS falls below a threshold. If a segment skips resampling, retain the old weights and multiply by the next incremental potential. Skipping resampling cannot be combined with assuming equal weights at the next segment.
Every run uses only particles, but we repeat independently many times. Is increasing only enough to eliminate finite- sampling bias?
You observed resampling and ancestry and read incremental ratios, telescoping potentials, the FK endpoint law, and finite-M error and degeneracy. Check ideal target construction separately from finite sampling performance.
Change the source: MCMC and SPT
“Back to my proposal: if copied drafts always produce the same image, could we preserve the generation rule and try new starts?”
Write the deterministic generator as : start produces . It includes the fixed generation computation; the same yields the same result. Originally the source has density , so we draw .
Every start has a queryable score . To favor high-scoring outputs, put the multiplier on the source:
The factor retains original relative source densities, while the exponential raises weights for starts yielding high scores. The generation rule stays fixed; the desired source distribution changes. Writing this target does not yet supply a sampler.
Use three discrete starts , each originally of probability , with and . The output multipliers are for and for .
The source weights become . After normalization, what are the probabilities of generating ? Add the probabilities of starts producing the same result, then compare with image-space base times reward.
You can also think first, then read the explanation. Your writing stays on this page and disappears when you reload.
Total source weight is , giving source probabilities . Only one start produces , so its probability is . Two produce , totaling .
The original output probabilities were . Multiplying by gives weights , again normalizing to . need not be one-to-one; include every source producing the same result.
More generally, also write the source normalizer as , the mean reward multiplier of base-generated outputs. Sampling from and applying the same gives any integrable image statistic the average
The original followed by produces the base output law. This equation is exactly its reward-weighted average. Source tilting therefore reweights the generator’s actual output law, without needing image-space density or eliminating original model error.
“We have a source target. Could we propose from the current , generate , score it, and decide whether to move there?”
Yes, but always accepting only higher scores again finds high-scoring points rather than specified proportions. A sequence of current and subsequent starts is a chain. Acceptance moves to ; rejection stays at . Staying still is also a step.
Using target-preserving Markov transitions to collect samples is Markov chain Monte Carlo (MCMC). First consider a symmetric proposal, equally likely in either direction. The acceptance probability is
Here is chain ’s reward strength. Normalizing constants cancel. The ratio compares prior density times reward multiplier. Draw ; move to if , otherwise keep . No reward gradient is needed.
Denote the chain target by and proposal density by . Symmetry means . Forward probability flow is
Exchanging gives the same reverse flow by symmetry. Including rejection probability at the current state prevents net target mass from flowing to one side. This equal-flow condition is detailed balance, giving target invariance.
An asymmetric proposal needs the additional ratio . The prior-and-reward-only version no longer suffices. Target preservation is a transition property; sufficient finite-run exploration is a separate question.
A fixed quantile generator maps standard-Normal sources to the same mixture base. Change only source acceptance and inspect lower-reward accepts, repeated positions, and the actual output histogram.
Predict, then inspect the distribution
Would always accepting only higher reward sample the target? Can the actual acceptance rule accept lower reward?
A uses λ = 0; B uses your λ. Both single chains start at z = -0.8 with identical iteration counts, proposal size, and seed.
The gray curve shows the original mixture. Running adds actual samples; a reference curve does not establish that it was sampled.
Explain using the plot and numbers
Acceptance uses both the standard-Normal prior and reward ratio and can accept lower reward. Repeated states after rejection must be retained; keeping only accepted moves changes the distribution. A finite chain can remain in one mode; 400 warm-up iterations do not certify convergence.
This explains the standard comparison. After changing controls or Python, use the settings and plots you actually ran.
Fixed G(z) = F_base^-1(Phi(z)): Phi is the standard-Normal CDF and F_base is the mixture CDF. Find z’s cumulative probability, then the location with the same cumulative probability in the base. This maps a Normal source to the mixture, always giving the same output for the same z. Numerical inversion does not train a model or differentiate reward. Acceptance uses prior × reward, and the histogram retains rejected and correlated states.
Change the parameters or the sampling rule. Python loads when you press Run and executes in your browser. Each run starts fresh; your edits are not saved.
The first opening imports the controls above; reopening preserves your edits. Settings are imported, not accumulated trained parameters or samples, so finite-sample plots need not match exactly.
The code implements this method and plots its distribution with Matplotlib. The first opening imports current controls; every run samples afresh. Change one parameter at the top. The native plot is not a record of edited Python output.
import math
import numpy as np
import matplotlib.pyplot as plt
# 1. SETTINGS -- change one, run again, and explain the plotted evidence.
strength = 2
steps = 1600
reward_kind = "two" # "right" or "two"
proposal_step = 0.35
rng = np.random.default_rng(42)
sigma = 0.55
grid = np.linspace(-6, 6, 481)
dx = grid[1] - grid[0]
# 2. FIXED MODEL -- analytic functions, no network training in this cell.
def pdf(x, mu=0.0, var=1.0):
return np.exp(-0.5 * (x-mu)**2 / var) / np.sqrt(2*np.pi*var)
def R(x):
return x if reward_kind == "right" else -(np.abs(x)-2.5)**2
base = 0.5*pdf(grid, -2, sigma**2)+0.5*pdf(grid, 2, sigma**2)
target = base*np.exp(strength*R(grid))
target /= target.sum()*dx # finite-grid quadrature reference
reference = target
reference_label = "Tilt reference (quadrature on [-6,6])"
extra_figure = None
# 3. SOURCE SEARCH -- a fixed quantile generator G; standard-Normal source prior.
def G(z):
p = min(1-1e-12, max(1e-12, 0.5*(1+math.erf(z/math.sqrt(2)))))
lo, hi = -8.0, 8.0
for _ in range(40):
mid = (lo+hi)/2
cdf = 0.25*(2+math.erf((mid+2)/(sigma*math.sqrt(2)))+math.erf((mid-2)/(sigma*math.sqrt(2))))
if cdf < p:
lo = mid
else:
hi = mid
return (lo+hi)/2
lambdas = [strength]
proposal_steps = [proposal_step] # step size is a separate choice
states = []
for _ in lambdas:
x = G(-0.8)
states.append([-0.8, x, float(R(x))])
samples, trace = [], []
accepted = swaps = lower_accepted = 0
burn = 400
for k in range(steps+burn):
for j, lam in enumerate(lambdas):
z, x, old_reward = states[j]
proposed_z = z+proposal_steps[j]*rng.normal()
proposed_x = G(proposed_z)
proposed_reward = float(R(proposed_x))
log_ratio = -0.5*(proposed_z**2-z**2)+lam*(proposed_reward-old_reward)
if math.log(max(1e-300, rng.random())) < min(0, log_ratio):
accepted += 1
lower_accepted += proposed_reward < old_reward
states[j] = [proposed_z, proposed_x, proposed_reward]
if len(states) > 1:
j = k % 2
log_ratio = (lambdas[j]-lambdas[j+1])*(states[j+1][2]-states[j][2])
if math.log(max(1e-300, rng.random())) < min(0, log_ratio):
states[j], states[j+1] = states[j+1], states[j] # swap z, G(z), reward together
swaps += 1
if k >= burn:
samples.append(states[-1][1]) # keep rejected/repeated states too
trace.append([state[0] for state in states])
samples = np.asarray(samples)
print("G and reward evaluations:", len(lambdas)*(steps+burn+1))
print("Accepted:", accepted, "accepted lower rewards:", lower_accepted, "swaps:", swaps)
print("Histogram includes rejections; states are correlated. Burn-in does not certify convergence.")
extra_figure, ax = plt.subplots(figsize=(8, 3.2))
for j, lam in enumerate(lambdas):
ax.plot(np.asarray(trace)[:,j], label=f"source chain, lambda={lam:g}", alpha=0.8)
ax.set(xlabel="retained iteration (rejections included)", ylabel="source z")
ax.legend(fontsize=8)
extra_figure.tight_layout()
# 4. PLOT AND CHECK -- the histogram is from the samples above.
fig, ax = plt.subplots(figsize=(8, 3.6))
ax.plot(grid, base, "--", color="gray", label="Original base")
ax.plot(grid, reference, ":", color="black", label=reference_label)
ax.hist(samples, bins=np.linspace(-6,6,49), weights=np.ones(len(samples))/(len(samples)*0.25), alpha=0.5, label="Actual generated samples")
ax.set(xlabel="clean x0", ylabel="density", xlim=(-6,6))
ax.legend(fontsize=8)
fig.tight_layout()
print("Outputs:", len(samples), "mean:", np.mean(samples), "right share:", np.mean(samples > 0))
print("Outside the plotted window:", np.sum(np.abs(samples)>6))
plt.show()The proposal is symmetric, old and new starts have equal prior density, and the new reward weight is half the old one. How should we decide?
Allowing lower-scoring candidates preserves the forward and reverse probability flows.
Always rejecting lower scores changes the rule into score climbing and loses the target-preservation argument.
The proposal is symmetric, not the target. Reward weights still affect acceptance.
“If high-scoring starts lie in distant regions, one chain may keep finding the same composition. Could several reward strengths search together and exchange what they find?”
Let chain use strength , targeting
A smaller is less influenced by reward and can be paired with larger proposal steps to explore other base-supported regions. A larger favors high scores and can be paired with smaller steps for refinement. Strength does not automatically determine proposal step size; they are separate design choices.
Each chain first tries local proposals with the acceptance rule, then proposes cross-chain swaps to bring exploratory results to the desired strength. This is Wang and colleagues’ Source Parallel Tempering (SPT): sample several source-space targets at different strengths, with swaps also subject to acceptance. Collect from the chain at desired strength , then output .
Two chains have strengths and starts . Their joint target is the product of their targets. Swapping the starts gives this reward-dependent new-to-old ratio:
Both sides contain the same , so prior factors cancel. Combine the exponential’s exponents. Which differences in strengths and rewards multiply together?
You can also think first, then read the explanation. Your writing stays on this page and disappears when you reload.
Subtract denominator exponents from numerator exponents, then factor common terms:
With symmetric selection of swap pairs, this rule preserves the product of chain targets. We cannot simply import another chain’s higher score: that may change the target as well as exploration speed.
One SPT round proposes local starts, generates and scores, accepts or stays using , then decides a pair’s swap using . Swap the start and its already computed reward together; do not pair a start with another image’s score.
Every fresh proposal costs generation and scoring. Samples in the desired chain may remain correlated, and we must check coverage of distinct high-reward regions. A target-preserving rule does not establish that finite search has reached the target.
Compare complete single-chain and three-chain records. Equal iterations per chain cost more G and reward calls with three chains. Inspect traces and coverage of both modes instead of treating higher mean reward as convergence.
Predict, then inspect the distribution
Can swaps between chains of different strengths reach another high-reward region? What extra generation and reward cost do they incur?
A is one chain at the target strength; B uses strengths 0, λ/2, λ with adjacent swaps. Iterations per chain match, so B has three times the evaluation cost; this is not an equal-budget comparison.
The gray curve shows the original mixture. Running adds actual samples; a reference curve does not establish that it was sampled.
Explain using the plot and numbers
Each chain makes an MCMC proposal, then swaps z, output, and reward together according to the swap ratio. Collect only the λ chain; do not mix in low-strength samples. Larger proposals are separately configured, not generated by λ. Finite runs can still explore poorly.
This explains the standard comparison. After changing controls or Python, use the settings and plots you actually ran.
The same fixed quantile generator. Proposal standard deviations are 1.4, 0.8, and your target-chain step size. Swap probability is min(1, exp[(λ_i−λ_j)(R_j−R_i)]). The trace shows source z across iterations, not diffusion time.
Change the parameters or the sampling rule. Python loads when you press Run and executes in your browser. Each run starts fresh; your edits are not saved.
The first opening imports the controls above; reopening preserves your edits. Settings are imported, not accumulated trained parameters or samples, so finite-sample plots need not match exactly.
The code implements this method and plots its distribution with Matplotlib. The first opening imports current controls; every run samples afresh. Change one parameter at the top. The native plot is not a record of edited Python output.
import math
import numpy as np
import matplotlib.pyplot as plt
# 1. SETTINGS -- change one, run again, and explain the plotted evidence.
strength = 2
steps = 1600
reward_kind = "two" # "right" or "two"
proposal_step = 0.35
rng = np.random.default_rng(42)
sigma = 0.55
grid = np.linspace(-6, 6, 481)
dx = grid[1] - grid[0]
# 2. FIXED MODEL -- analytic functions, no network training in this cell.
def pdf(x, mu=0.0, var=1.0):
return np.exp(-0.5 * (x-mu)**2 / var) / np.sqrt(2*np.pi*var)
def R(x):
return x if reward_kind == "right" else -(np.abs(x)-2.5)**2
base = 0.5*pdf(grid, -2, sigma**2)+0.5*pdf(grid, 2, sigma**2)
target = base*np.exp(strength*R(grid))
target /= target.sum()*dx # finite-grid quadrature reference
reference = target
reference_label = "Tilt reference (quadrature on [-6,6])"
extra_figure = None
# 3. SOURCE SEARCH -- a fixed quantile generator G; standard-Normal source prior.
def G(z):
p = min(1-1e-12, max(1e-12, 0.5*(1+math.erf(z/math.sqrt(2)))))
lo, hi = -8.0, 8.0
for _ in range(40):
mid = (lo+hi)/2
cdf = 0.25*(2+math.erf((mid+2)/(sigma*math.sqrt(2)))+math.erf((mid-2)/(sigma*math.sqrt(2))))
if cdf < p:
lo = mid
else:
hi = mid
return (lo+hi)/2
lambdas = [0.0, strength/2, strength]
proposal_steps = [1.4, 0.8, proposal_step] # step size is a separate choice
states = []
for _ in lambdas:
x = G(-0.8)
states.append([-0.8, x, float(R(x))])
samples, trace = [], []
accepted = swaps = lower_accepted = 0
burn = 400
for k in range(steps+burn):
for j, lam in enumerate(lambdas):
z, x, old_reward = states[j]
proposed_z = z+proposal_steps[j]*rng.normal()
proposed_x = G(proposed_z)
proposed_reward = float(R(proposed_x))
log_ratio = -0.5*(proposed_z**2-z**2)+lam*(proposed_reward-old_reward)
if math.log(max(1e-300, rng.random())) < min(0, log_ratio):
accepted += 1
lower_accepted += proposed_reward < old_reward
states[j] = [proposed_z, proposed_x, proposed_reward]
if len(states) > 1:
j = k % 2
log_ratio = (lambdas[j]-lambdas[j+1])*(states[j+1][2]-states[j][2])
if math.log(max(1e-300, rng.random())) < min(0, log_ratio):
states[j], states[j+1] = states[j+1], states[j] # swap z, G(z), reward together
swaps += 1
if k >= burn:
samples.append(states[-1][1]) # keep rejected/repeated states too
trace.append([state[0] for state in states])
samples = np.asarray(samples)
print("G and reward evaluations:", len(lambdas)*(steps+burn+1))
print("Accepted:", accepted, "accepted lower rewards:", lower_accepted, "swaps:", swaps)
print("Histogram includes rejections; states are correlated. Burn-in does not certify convergence.")
extra_figure, ax = plt.subplots(figsize=(8, 3.2))
for j, lam in enumerate(lambdas):
ax.plot(np.asarray(trace)[:,j], label=f"source chain, lambda={lam:g}", alpha=0.8)
ax.set(xlabel="retained iteration (rejections included)", ylabel="source z")
ax.legend(fontsize=8)
extra_figure.tight_layout()
# 4. PLOT AND CHECK -- the histogram is from the samples above.
fig, ax = plt.subplots(figsize=(8, 3.6))
ax.plot(grid, base, "--", color="gray", label="Original base")
ax.plot(grid, reference, ":", color="black", label=reference_label)
ax.hist(samples, bins=np.linspace(-6,6,49), weights=np.ones(len(samples))/(len(samples)*0.25), alpha=0.5, label="Actual generated samples")
ax.set(xlabel="clean x0", ylabel="density", xlim=(-6,6))
ax.legend(fontsize=8)
fig.tight_layout()
print("Outputs:", len(samples), "mean:", np.mean(samples), "right share:", np.mean(samples > 0))
print("Outside the plotted window:", np.sum(np.abs(samples)>6))
plt.show()SPT generates and scores again for each new proposed start. Can all these operations preserve the generative parameters ?
You implemented MCMC and SPT and read how source tilting yields output laws, acceptance and swap derivations, finite-search limits, and generation cost. Return before comparing all methods under different constraints.
The method names are now familiar. Place each action and limitation where it belongs. Read each row’s explanation before continuing.
You can save noisy states, advance each by a segment, and cheaply estimate future multipliers. Which operation directly reallocates computation before completion?
These are SMC’s interface prerequisites; estimator quality and degeneracy still require checks.
This spends every candidate’s full generation cost before choosing.
This overcounts the preference; use incremental ratios consistent with the target.
This generator is deterministic: the same draft always yields the same future. You want to keep its generation rule and explore different answers. Which approach fits?
Identical states following the same deterministic rule yield the same image, regardless of copy count.
Fixed G does not require fixed z. Source sampling can change while preserving a specified source target.
One high-scoring point establishes neither the proportions of other results nor coverage of other high-scoring regions.
So far, scores decide which candidates stay. Another extension keeps every draft and adjusts its next prediction toward the best candidate’s predicted completion. It also avoids differentiating reward, but differs from SMC resampling.
At time , retain states , compute their denoised estimates , score them, and select
We score predicted completions, not endpoint images . For candidate , the vector points from its prediction toward the reference prediction. Kim and colleagues call guidance of other particles by the best candidate Best-of- Guidance (BoNG). Here we keep for the intermediate candidate count.
How does that displacement convert to noise prediction? Fix current and use VP denoising:
Subtracting the two denoised estimates cancels fixed . To make the left-hand displacement equal , rearrange, assuming and :
Here controls displacement strength. At , the original prediction remains. At , the new denoised estimate at the fixed current state equals the reference, without implying the same next state or final image. The original sampler must use the combined prediction to advance to a lower noise level.
At each reverse step, estimate clean images, score and choose a reference, modify every prediction, and continue. The best index may change. The reference candidate’s own displacement is zero. Candidate differences supply directions without computing or updating .
Unlike FK, this has not established terminal weighting equal to , so it does not directly prove exact tilted sampling. The source note also treats it as an extension. This VP conversion retains ; do not mix it with a schedule whose signal amplitude is .
The main text gives terminal IW, SMC’s segment loop, and SPT’s acceptance and swap rules. The source skill manual additionally writes them as mathematical pipelines and pseudocode for stepwise reference.
An evaluator that only gives scores can still steer: sample completed candidates by weight, resample drafts with incremental weights, or keep a deterministic generator and change source sampling. Finding a high-scoring image and approximating specified proportions require different checks. More computation does not automatically make them equivalent.
This overview connects terminal selection, intermediate resampling, and source search. SPT rows are reward strengths, not diffusion time. Its generator is our fixed , while FK’s is a different incremental weight function. The mechanism drawing does not establish exact finite-computation sampling of .
Select the image to enlargeTo examine sampling rules, we assumed a base of . If the model actually outputs , the same terminal weighting approximates the normalized distribution with weights
With a finite-step sampler, the base is more precisely its actual output law, including discretization error. More candidates, particles, or chain exploration can improve approximation relative to that base, without automatically correcting it to .
A result with no base support cannot appear through finite-reward reweighting. Even amplified low-probability regions may remain unseen by finite candidates. Model error, sampling approximation, reward, plausibility, and diversity therefore remain distinct checks.
Next, consider a more concrete request: the user supplies a blurry photo and wants a sharp image truly compatible with it. How should the evaluator score restoration of that photo, rather than simply a nice-looking dog? U7.4 continues from the measurement process.
Read the same proportions, FK/SMC derivation, SPT acceptance rules, and BoNG extension continuously in U7.3 What If the Reward Scores Samples but Gives No Direction?.