Probability describes uncertainty under a model. It does not establish causation: correlated variables can have a strong conditional relationship without either causing the other.
Random Variables and Moments #
A random variable maps outcomes to values. Its distribution assigns probability mass or density to those values. For a scalar random variable \(X\), expectation and variance are:
\[ \mathbb{E}[X], \qquad \operatorname{Var}(X) = \mathbb{E}[(X-\mathbb{E}[X])^2]. \]Standard deviation is the square root of variance. For a random vector \(X\), the mean is a vector and covariance is a matrix:
\[ \operatorname{Cov}(X) = \mathbb{E}[(X-\mu)(X-\mu)^\mathsf{T}]. \]Covariance measures linear co-variation under the distribution; it does not by itself identify a causal mechanism.
Conditional Probability and Bayes’ Rule #
For \(P(B) > 0\),
\[ P(A \mid B) = \frac{P(A \cap B)}{P(B)}. \]Bayes’ rule reverses a conditional relationship:
\[ P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}. \]Machine-learning classifiers commonly estimate \(P(y \mid x)\). Such estimates require calibration and validation; a score presented as a probability is useful only when empirical frequencies support that interpretation.
Common Distributions #
| Distribution | Support | Parameters and Use |
|---|---|---|
| Bernoulli | \(0, 1\) | Probability \(p\) for one binary trial |
| Binomial | integers from \(0\) to \(n\) | Number of successes in \(n\) independent Bernoulli trials |
| Geometric | positive integers | Trials until the first success |
| Poisson | nonnegative integers | Event count in an interval at rate \(\lambda\) |
| Exponential | nonnegative reals | Waiting time for a Poisson process |
| Normal | real numbers | Mean \(\mu\) and variance \(\sigma^2\) |
| Gamma | positive reals | Waiting-time and positive-scale modeling |
| Beta | \([0, 1]\) | Distribution over a probability parameter |
| Multinomial | count vectors | Counts across mutually exclusive categories |
The law of large numbers states that averages of suitable independent, identically distributed observations converge toward the expected value as sample size grows. It does not make a small sample representative, cause outcomes to alternate, or repair systematic sampling bias.
Comments
Read or comment on GitHub (sign-in required to comment)