🏠
Guest Not signed in

First-step analysis: condition on the first transition

Condition on what happens on the first move. Everything after is a smaller copy of the same problem β€” solve recursively.

Method · First Step Analysis
Prereqs: Markov Chain
Intro
Five-state random walk on the integers -2 to 2 with absorbing boundaries β€” the canonical gambler's ruin / first-step-analysis setup.
User:Bender2k14, CC BY-SA 3.0 · CC-BY-SA-3.0 · Wikimedia Commons

When a process wanders between states and stops when it hits one of a few “sinks” (gambler’s ruin, random walks on a graph, “flips until pattern X”), don’t track every path. Define $f(i)$ = the answer starting from state $i$. At each interior state, $f$ satisfies one simple equation: take one step, then average $f$ over where you land. We’ll show this on a small bankroll trying to reach a target.

βœ“ Intro Β· expand
Independent · Legal