Stochastic-process theory offers two entirely different ways to observe a system: one imagines many parallel systems and observes their distribution at a fixed time; the other observes only a single system, but lets it run for a very long time. When do these two kinds of averages give the same result? That is the question ergodicity tries to answer.

1. Ensemble Averages and Time Averages

1.1 Ensemble average

Suppose there are many mutually independent systems, observed at the same time tt: Xt(1),Xt(2),…X_t^{(1)},X_t^{(2)},\dots. Averaging across these systems approaches the expectation E[Xt]E[X_t]; this kind of average across different realizations is called an ensemble average.

1.2 Time average: the central question

If there is only one system but it is observed for a long time, we can compute XˉT=1T∑t=1TXt\bar X_T=\frac1T\sum_{t=1}^T X_t; this is called a time average. The central question is: does the time average approach the ensemble average? If it does in an appropriate sense, the process is ergodic.

2. Why This Is a Deep Question: A Concrete Non-Ergodic Example

Suppose a system starts by randomly choosing, with probability 1/21/2 each, to enter region A or region B, and then stays there forever: entering A means its long-run value is permanently 0, entering B means it is permanently 10. Across the ensemble, E[X]=0.5×0+0.5×10=5E[X]=0.5\times0+0.5\times10=5, and the ensemble variance across different systems is E[X2]−E[X]2=0.5×0+0.5×100−25=25E[X^2]-E[X]^2=0.5\times0+0.5\times100-25=25, a standard deviation of 5 — if you only looked at the ensemble statistics, you would think the typical value “fluctuates around 5 by about 5.” But the long-run average of any single sample path can only be exactly 0 or exactly 10, never close to 5, and the variance of the time average within a single path is 0 (once it is decided whether the system enters A or B, there is no further randomness at all). This is exactly the core consequence of ergodicity failing: observing one path, no matter how long, can never reveal that the true ensemble variance is 25, because a single path is locked into one of the two extremes from the very start. This system has a textbook non-ergodic structure.

3. Ergodicity in Stationarity and Markov Chains

3.1 Stationarity is not the same as ergodicity

Stationarity says “the probability law does not change under a shift in time”; ergodicity asks “can one sufficiently long trajectory explore the entire statistical structure?” A process can be stationary without being ergodic — the example in Section 2 only needs a slight adjustment (for instance, making the two constant values A and B themselves not change distribution over time) to become simultaneously stationary and non-ergodic. Do not conflate “a stable long-run statistical law” with “one trajectory can represent the whole ensemble.”

3.2 Ergodicity in a Markov chain, worked out

For many finite-state Markov chains, if the chain is irreducible, aperiodic, and has a stationary distribution π\pi, the long-run proportion of time converges to that stationary distribution: 1T∑t=1T1(Xt=i)→πi\frac1T\sum_{t=1}^T\mathbf1(X_t=i)\to\pi_i — the proportion of time a long trajectory spends in state ii approaches the probability πi\pi_i in the stationary distribution. This is a textbook ergodic theorem. Continuing with the stationary distribution π=(1/3,1/2,1/6)\pi=(1/3,1/2,1/6) solved in Stochastic Processes 6: if this chain is actually simulated for T=12,000T=12{,}000 steps, state B should appear close to 12,000×0.5=6,00012{,}000\times0.5=6{,}000 times, and state C should appear close to 12,000×1/6=2,00012{,}000\times1/6=2{,}000 times — there is no need to run many chains in parallel and take an ensemble average; one sufficiently long trajectory is enough on its own to estimate the stationary distribution, which is a direct demonstration of this Markov chain being ergodic.

4. Ergodic Theorems, Mixing, and Non-Ergodic Systems

4.1 The Birkhoff ergodic theorem and mixing

A more general ergodic theorem tells us that, under suitable conditions, 1T∑t=1Tf(Xt)\frac1T\sum_{t=1}^T f(X_t) can converge to E[f(X)]E[f(X)], meaning that a theoretical ensemble expectation can in principle be estimated from a single long time series — extremely important for practical data analysis, since in reality we often have only “one history of one world.” Mixing is a stronger notion of “forgetting the past” than ergodicity: roughly speaking, a system exhibits mixing behavior if its distant-future state becomes increasingly close to independent of its current state. Mixing often implies good ergodic behavior, but the two are not fully equivalent.

4.2 Why non-ergodic systems matter

Non-ergodicity commonly appears in dynamical systems with multiple attractors, Markov chains with multiple communicating classes or absorbing states, complex physical systems such as glassy states, and path-dependent systems with long-term lock-in behavior — in these cases a single trajectory may forever explore only part of the state space. The example in Section 2 may look like a deliberately contrived extreme case, but the “multiple attractor” structure found in real systems (for example, a training process converging to different local optima, or an ecosystem settling into different equilibria) is essentially the same phenomenon: running the system once and observing it for a long time does not necessarily produce a conclusion that represents every possible initial condition or source of randomness.

5. Summary

What ergodicity really asks is: is observing one system for long enough equivalent to observing many parallel systems? The non-ergodic example in Section 2 and the ergodic Markov chain example in Section 3.2 form a sharp contrast: the former can never learn the true ensemble variance no matter how long it is observed, while the latter can accurately estimate the entire stationary distribution from one sufficiently long trajectory. Ergodicity connects probability distributions, long-run averages, and sample paths, and is the central concept for deciding whether one time series can represent the whole ensemble — a premise that must be confirmed, not assumed, in nearly every situation where statistical inference is drawn from a single observed history.