Probability and Statistics 1: Sample Spaces, Events, and Random Variables

Probability theory studies uncertainty, but “uncertain” doesn’t mean unanalyzable. A probability model first lists possible outcomes, then describes how likely different outcomes are with numbers. Statistics, in turn, works backward from already-observed data to infer a model, estimate unknown parameters, or judge whether some hypothesis is credible.

Beginners often memorize combinatorics or distribution formulas directly, without clearly distinguishing samples, events, random variables, and observed values. Mixing up these concepts makes conditional probability, expectation, statistical inference, and stochastic processes hard to understand. This article first builds a vocabulary that can be reused repeatedly, then works through calculations using examples like rolling dice, quality inspection, and server requests.

1. From Random Experiments to a Probability Space

1.1 Random experiments and sample spaces

An operation whose result can’t be determined before it’s carried out can be treated as a random experiment. Typical examples include rolling a die and recording the number, drawing a product and checking whether it passes inspection, recording the number of requests a server receives in the next minute, and measuring an electronic component’s lifespan. A probability model needs to clearly define “one trial” and “the outcome to record” — a vaguely defined experiment can’t produce a well-defined probability.

The set of every possible outcome of a random experiment is called the sample space, usually written $\Omega$; a single outcome in the sample space is written $\omega$. The sample space of rolling a six-sided die is $\Omega=\{1,2,3,4,5,6\}$. When rolling two distinguishable dice, $\Omega=\{(i,j):i,j\in\{1,2,3,4,5,6\}\}$ represents 36 ordered outcomes. $(1,6)$ and $(6,1)$ are different outcomes, even though both sum to 7.

A sample space can also be an infinite set. Repeatedly flipping a coin until the first heads can be represented with $\{1,2,3,\dots\}$ for the waiting count; an electronic component’s lifespan can be represented with $[0,\infty)$; a temperature curve over a period of time needs to be represented with a set of functions instead. A finite sample space is suited to counting methods, while a continuous or function-valued sample space needs tools like density, integration, and measure.

1.2 Events and set operations

An event is a subset of the sample space. When rolling a die, let $A=\{2,4,6\}$ represent “the number is even,” and let $B=\{4,5,6\}$ represent “the number is greater than 3.” The observed outcome falling within the event’s set means the event occurred.

Set notation	Event meaning
$A\cap B$	$A$ and $B$ both occur
$A\cup B$	at least one of $A$ or $B$ occurs
$A^c$	$A$ does not occur
$A\setminus B$	$A$ occurs but $B$ does not
$A\cap B=\varnothing$	$A$ and $B$ cannot occur simultaneously

In the example above, $A\cap B=\{4,6\}$, so “even and greater than 3” contains the numbers 4 and 6.

For a finite sample space, every subset can generally be treated as an event. In an infinite sample space, pathological sets exist that can’t be consistently assigned a probability, so a rigorous probability model specifies a family of event sets $\mathcal F$. $\mathcal F$ is closed under complementation and countable union, and is called a $\sigma$-algebra. At an introductory level, $\mathcal F$ can be understood as the collection of events the model allows you to ask about and assign a probability to.

1.3 Probability measures and the equally-likely assumption

A probability measure $P$ maps an event $A\in\mathcal F$ to a value within $[0,1]$. Kolmogorov’s axioms require nonnegativity $P(A)\ge 0$, normalization $P(\Omega)=1$, and countable additivity over mutually exclusive events. The triple $(\Omega,\mathcal F,P)$ is called a probability space.

The axioms let us derive $P(A^c)=1-P(A)$ and $P(A\cup B)=P(A)+P(B)-P(A\cap B)$. The union formula subtracts the intersection because directly adding $P(A)$ and $P(B)$ would double-count the intersection.

If the elementary outcomes of a finite sample space are equally likely, $P(A)=|A|/|\Omega|$ can be used. The 36 ordered outcomes of two fair dice are equally likely, but the sums of the two dice are not. A sum of 7 has six outcomes — $(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)$ — so $P(D_1+D_2=7)=6/36=1/6$; a sum of 2 has only one outcome, $(1,1)$. Treating the sums from 2 to 12 as 11 equally likely outcomes gives the wrong answer.

Counting formulas can’t be applied to a biased die, components with different failure rates, or continuous measurements. Before using combinatorics, you must first confirm that the finiteness and equally-likely assumptions actually hold.

2. Random Variables and Distributions

2.1 A random variable is a function

A random variable isn’t a symbol that changes randomly — it’s a function that maps from the sample space to a set of numbers, written $X:\Omega\rightarrow\mathbb R$. In the two-dice experiment, we can define $X(i,j)=i+j$. The elementary outcome is $(2,5)$, and the random variable’s observed value is $X(2,5)=7$.

Multiple random variables can be defined on the same sample space — for example, the sum of the dice, the maximum of the two, the difference between them, or whether the first die is even. A random variable lets a researcher ignore irrelevant details of the trial and keep only the quantity the question actually cares about.

2.2 Discrete variables, continuous variables, and the CDF

A discrete random variable has finitely or countably many possible values — for example, a die roll, a request count, or a failure count. A discrete variable uses a probability mass function (PMF), $p_X(x)=P(X=x)$.

If $X$ is the sum of two fair dice, then $p_X(2)=1/36$ and $p_X(7)=6/36$. Computing the PMF requires summing over every elementary outcome that produces each possible value, and the sum of every PMF value must equal 1.

A continuous random variable usually takes values across an interval — for example, time, temperature, or voltage. A continuous variable commonly uses a probability density function (PDF), $f_X(x)$, with interval probabilities given by $P(a\le X\le b)=\int_a^b f_X(x)\,dx$. The probability of a continuous variable taking any single point is usually 0; a density value can exceed 1 — what’s actually bounded within $[0,1]$ is the probability represented by the area under the curve.

Any real-valued random variable can use a cumulative distribution function (CDF), $F_X(x)=P(X\le x)$. The CDF is monotonically nondecreasing, and satisfies $\lim_{x\to-\infty}F_X(x)=0$ and $\lim_{x\to\infty}F_X(x)=1$. A discrete variable’s CDF looks like a staircase; if a continuous variable has a PDF, then $F_X(x)=\int_{-\infty}^{x}f_X(u)\,du$.

3. Conditional Probability, Independence, and Bayes’ Theorem

3.1 Conditional probability narrows the sample range

Given that event $B$ has already occurred, the conditional probability of event $A$ is defined as $P(A\mid B)=P(A\cap B)/P(B)$, where $P(B)>0$. Conditional probability restricts the analysis to within $B$, then renormalizes the probability using $P(B)$.

Using two fair dice as an example: let $A$ represent “the sum is 8,” and let $B$ represent “the first die is even.” Event $B$ contains 18 equally likely outcomes; the outcomes satisfying both $A$ and $B$ are $(2,6),(4,4),(6,2)$, so $P(A\mid B)=3/18=1/6$. Without knowing the parity of the first die, a sum of 8 has five outcomes, giving $P(A)=5/36$. The extra piece of information changed the comparable set of outcomes, and therefore changed the event’s probability too.

The multiplication rule $P(A\cap B)=P(A\mid B)P(B)$ can be extended to multiple events. For a sequence of events $A_1,\dots,A_n$, their joint probability can be decomposed into a product of successive conditional probabilities. Autoregressive models in machine learning use the same chain rule, breaking a token sequence’s joint probability into the probability of each token conditioned on the tokens before it.

3.2 Independence is a condition on the joint distribution

Events $A$ and $B$ are independent when $P(A\cap B)=P(A)P(B)$. If $P(B)>0$, an equivalent condition is $P(A\mid B)=P(A)$: learning that $B$ occurred doesn’t change the probability of $A$.

Mutual exclusivity and independence have different meanings. Two mutually exclusive events, each with positive probability, cannot occur together, so $P(A\cap B)=0$, yet $P(A)P(B)>0$ — mutually exclusive events are therefore not independent. Independence also can’t just be declared by intuition; it must come from the sampling design, the generating mechanism, or a testable assumption about the joint distribution.

For multiple events, pairwise independence only requires that every pair’s joint probability factors; mutual independence additionally requires that the joint probability of any subset factors. Mutual independence always implies pairwise independence, but the converse generally doesn’t hold.

3.3 The law of total probability and Bayes’ theorem

If $B_1,\dots,B_m$ are pairwise mutually exclusive and their union is the entire sample space, then the probability of event $A$ can be split by source: $P(A)=\sum_iP(A\mid B_i)P(B_i)$. This is called the law of total probability.

Bayes’ theorem reverses the direction of conditioning: $P(B_j\mid A)=P(A\mid B_j)P(B_j)/P(A)$. The numerator is made up of the likelihood $P(A\mid B_j)$ and the prior probability $P(B_j)$; the denominator sums the joint probability over every possible source, making the posterior probabilities sum to 1.

Suppose a disease has a 1% prevalence, a test has a 95% true-positive rate for patients, and a 5% false-positive rate for non-patients. Let $D$ represent having the disease, and $+$ represent a positive test. The overall probability of a positive result is $P(+)=0.95\times0.01+0.05\times0.99=0.059$, so the probability of actually having the disease given a positive result is $P(D\mid +)=0.95\times0.01/0.059\approx0.161$. Even with a highly sensitive test, someone testing positive only has about a 16.1% chance of actually being sick — because there are far more non-patients than patients, and a 5% false-positive rate accumulates into a large number of positive results.

A Bayes calculation needs to keep both the base rate and the test’s conditional probability in view at the same time. Looking only at $P(+\mid D)$ and mistaking that 95% figure for $P(D\mid +)$ is a common mix-up of the direction of conditioning.

4. Summary: A Probability Space Is the Shared Language for Later Topics

A probability space $(\Omega,\mathcal F,P)$ defines the possible outcomes, the events that can be asked about, and the rules governing probability; a random variable $X$ then maps elementary outcomes into numbers that can be analyzed. The PMF, PDF, and CDF describe the distribution of a single random variable, while conditional probability, independence, and Bayes’ theorem describe how multiple events affect each other and how probability judgments get updated given new evidence. This vocabulary — sample space, event, probability measure, and random variable — is the foundation that gets reused repeatedly when later discussing comparing distributions, joint distributions, and statistical estimation.

From here, you can continue on to Stochastic Processes 1: What Is a Stochastic Process?, which extends a random variable into an indexed family of random variables; for a counting-based application, see Stochastic Processes 9: The Poisson Process.