Stochastic Processes 8: Hitting Times and Stopping Times

The preceding stochastic-process topics mainly studied “what the state is” and “how the state changes,” but many practical questions are not about a fixed time point but about “when does something first happen”: when does a random walk first return to the origin, when does a stock price first exceed a threshold, when does a queueing system first empty out, when does a gambler go broke. These questions lead to three concepts: hitting time, first-passage time, and stopping time.

1. Hitting Times

1.1 The arrival time itself is a random variable

Let $\{X_t\}$ be a stochastic process, and given a target set $A$, define $\tau_A=\inf\{t\ge0:X_t\in A\}$ as the first time the process enters $A$; if $A=\{a\}$ is a single state, write it as $\tau_a$. For example, the first time a one-dimensional random walk reaches position 10 is $\tau_{10}=\inf\{t\ge0:X_t=10\}$. $\tau_A$ is not a fixed time but a random variable: different sample paths may arrive at step 20, at step 300, or never arrive at all, so we can study $P(\tau_A<\infty)$ (whether it eventually arrives), $P(\tau_A\le t)$ (the probability of arriving before time $t$), and $E[\tau_A]$ (the average time required). A stochastic process produces not only random states but also random times.

1.2 First-passage time and return time

When the target is the first crossing of some threshold, this is usually called a first-passage time — for example, $\tau=\inf\{t:X_t\ge b\}$ is the first time the process reaches or exceeds threshold $b$. When the question is the first return to the initial state, it is called a return time — for example, for a random walk starting at the origin, the first time it returns to the origin again is $\tau_0^+=\inf\{t\ge1:X_t=0\}$, which is exactly the time that Stochastic Processes 7’s recurrent/transient determination depends on.

2. Stopping Times

2.1 No peeking into the future

Not every “random time” is called a stopping time. The core requirement of a stopping time is: at the moment time $t$ arrives, whether to stop must be decidable using only the information observed so far. Let $\mathcal F_t$ denote all information available up to time $t$; if the event $\{\tau\le t\}$ can be determined using only $\mathcal F_t$, then $\tau$ is a stopping time. Intuitively, this means you cannot peek into the future.

2.2 A counterexample that is not a stopping time

“The last time a random walk visits the origin” is usually not a stopping time: when you are currently at the origin, you cannot decide, using only the information available now, whether “this is the last visit” — confirming this requires first seeing whether the process ever returns at any future time. It is a well-defined random time, but it fails the condition that stopping can be decided using only present information, so it is not a stopping time.

3. Application: The Stopping Time of Gambler’s Ruin

3.1 The stopping rule is itself a stopping time

Let a gambler’s wealth be $X_t$, where each round wins or loses 1 unit with equal probability, and the game ends once the wealth reaches 0 or $N$; the corresponding stopping time is $\tau=\inf\{t:X_t\in\{0,N\}\}$. Deciding “should we stop now” only requires checking whether the current wealth falls in $\{0,N\}$ — no peeking into the future is needed — so $\tau$ is a valid stopping time. Around this $\tau$ we can study the probability of reaching $N$ first, the probability of reaching 0 first, and the mean number of rounds before the game ends.

3.2 Worked out: the mean number of rounds

Continuing the $k=30$, $N=100$ example from Stochastic Processes 7: for the Gambler’s Ruin problem with a fair coin, the mean stopping time has a simple formula, $E[\tau]=k(N-k)$. Plugging in $k=30$, $N=100$ gives $E[\tau]=30\times70=2100$ — on average it takes 2100 rounds to decide a winner, far longer than the intuition suggested by the result “the probability of reaching 100 first is 0.3.” When $k$ is moved to the middle ($k=50$), $E[\tau]=50\times50=2500$ reaches its maximum: a starting point far from both boundaries takes longer on average to hit either one; the closer the starting point is to one boundary ($k$ close to 0 or $N$), the smaller $E[\tau]$, because the nearby boundary is hit quickly.

4. Stopping Times and the Strong Markov Property

Many real-world decisions take the form of “stopping once a certain state is observed”: selling once a price reaches a target, repairing a machine after it fails, stopping an experiment once a test statistic exceeds a threshold, ending a busy period once the queue length drops to zero — none of these are fixed times, but random times determined by the stochastic process itself, which is also why stopping times keep appearing in financial mathematics, reliability analysis, and statistical testing. For a Markov process, once it reaches a certain state, the future generally only needs to be analyzed afresh from that current state; extending this property to random times gives the strong Markov property: the Markov property holds not only at fixed times, but restarting at any valid stopping time, the future still depends only on the state at the moment of stopping, not on the path taken to reach that state. This is exactly the theoretical basis that lets Section 3.2 treat “after reaching 0 or $N$” as a fresh starting point for analysis, and it is also what makes hitting-time analysis decomposable recursively.

5. Common Misconceptions and Summary

The easiest confusion is treating “random time” and “stopping time” as synonyms: the difference lies entirely in whether the decision can be made using only present information — the counterexample in Section 2.2 is a reminder that as long as the decision requires peeking into the future, no matter how naturally the time point is defined, it is not a stopping time. Another common misconception is thinking that a hitting-time probability result (for example, which boundary is reached first) already fully answers the question, while overlooking how long it takes on average: the example in Section 3.2 shows that the probability of reaching a goal and the time needed to reach it are two independent questions, and neither alone is enough to draw a conclusion.

A hitting time changes the question from “what state is it in at a fixed time” to “when does a random event first occur”; a stopping time further requires that the decision to stop can depend only on present and past information. Together, they are foundational tools shared by random walks, Markov chains, queueing theory, financial mathematics, and stochastic control.