M4.1 No Way Back: Absorbing States and Absorption Times

A game that ends by itself

After each round, a ten-sided die ends a single-player game on zero. Otherwise another round begins. There is no revival within that game.

Knowing that the game eventually ends leaves several questions:

1. How many rounds does a game last on average?
2. After surviving twenty rounds, how many additional rounds remain on average?
3. By which round have at least half the games ended?

Identical fresh dice suggest the next-round chance does not depend on elapsed rounds. But computing a mean and computing a halfway probability are different tasks. Define the ending time as a random variable first.

Tokens falling into a pit

Start a thousand tokens alive. About a hundred fall into the ended-state pit in round one, ninety more in round two, and so on. The number falling decreases while the fraction of survivors falling stays at one tenth.

A surviving token faces the same next-round rule regardless of its age. Twenty completed rounds do not consume a quota: expected remaining lifetime is still ten. This is memorylessness, not compensation for earlier luck.

A constant hazard gives a geometric tail. Its median is seven rounds; rare long games raise its mean to ten.

Turn the picture into equations

A geometrically decaying tail

Surviving $n$ rounds requires avoiding the ending outcome each time:

This is a geometric distribution, counting trials through the first success, where success means ending. Equivalently, $P^n(\mathrm A,\mathrm A)=(1-q)^n$.

Expected time by first-step analysis

Let $m$ be expected remaining rounds. Spend one round, then end with probability $q$ or return to the same future problem with probability $1-q$:

This is first-step analysis. For each transient state $x$, expected absorption time satisfies

The unknown expectations form a linear system. The Markov property makes the remaining future depend only on the next state. Distribution propagation looks forward; this recursion propagates future expectations backward.

Median and variance

For $q=0.1$, the smallest $n$ with $(1-q)^n\le1/2$ is seven. Thus at least half end by the seventh round. Variance is 90 and standard deviation about 9.5. The right tail raises the mean above the median.

Why stationarity is less informative here

The only stationary law in this two-state example is $(0,1)$. It identifies the final state without answering how long arrival takes. Absorption breaks irreducibility; hitting times and route probabilities become more informative questions.

Revisit the lifetime questions

With constant hazard $q=0.1$ and no return, mean lifetime is ten, remaining mean after twenty surviving rounds is ten, and median lifetime is seven. The identity $\Pr(\tau>20+k\mid\tau>20)=(1-q)^k$ explains the unchanged remaining law.

If hazard increases with age, memorylessness fails. Nondecreasing hazards starting at $q_1$ give lifetime no longer on average than the constant-$q_1$ model. If the ended state returns with probability $r>0$, it is no longer absorbing; its long-run share becomes $q/(q+r)$. First hitting time still makes sense separately.

Try it

import numpy as np
rng = np.random.default_rng(0)
q, T = .1, 100000
tau = rng.geometric(q,size=T)
print('mean, median, std:',tau.mean(),np.median(tau),tau.std())
print('remaining after 20:',(tau[tau>20]-20).mean())
def play():
    n = 1
    while rng.random()>=min(1,.1+.02*(n-1)):
        n += 1
    return n
tau2 = np.array([play() for _ in range(T)])
print('increasing hazard:',tau2.mean(), 'remaining after 5:',(tau2[tau2>5]-5).mean())

Check your understanding

References

1. Norris, J. R. Markov Chains. Cambridge University Press, 1997.
2. Grimmett, G., Stirzaker, D. Probability and Random Processes, 3rd ed. Oxford University Press, 2001.
3. Kemeny, J. G., Snell, J. L. Finite Markov Chains. Springer, 1976.