Fundamentals of Probability and Statistics

Probability is the mathematical language of uncertainty, helping us model and reason about unpredictable outcomes.

Theoretical probability uses ideal models; empirical probability relies on real-world data and observations.

Classical, frequentist, and Bayesian interpretations offer different ways to define and apply probability.

A sample space lists all outcomes; events are subsets of outcomes we care about.

Basic axioms include non-negativity, normalization, and additivity for mutually exclusive events.

Independence means events don't affect each other. Expectation is the average outcome over many trials.

These measure how much outcomes deviate from the average.

Covariance and correlation measure how two variables move together, helping identify relationships in data.

Markov, Chebyshev, and Jensen's inequalities help bound probabilities when distributions are unknown.

Union, intersection, and complement help describe how events relate.

Permutations and combinations are tools to count possible outcomes.

Tools like multinomials, Catalan numbers, and recurrence relations solve complex counting problems.

Conditional probability quantifies the chance of one event given another.

Bayes' Rule updates beliefs using prior knowledge and new evidence.

A random variable maps outcomes to numerical values for analysis.

Discrete variables take countable values; continuous ones can take any value in a range.

The PDF and CDF describe how probabilities are distributed for continuous variables, key for computing expected values and ranges.

Bernoulli models one trial; binomial models multiple independent trials.

Poisson models rare events over time; geometric models trials until the first success.

Negative binomial tracks successes; hypergeometric handles sampling without replacement.

Uniform gives equal likelihood in an interval; normal is bell-shaped and central to stats.

Exponential handles time between events; gamma generalizes it; beta models probabilities.

The CLT explains why averages of samples tend to follow a normal distribution.

Confidence intervals estimate ranges for unknown parameters with a set level of confidence.

Hypothesis testing uses sample data to assess claims about populations.

P-values quantify how compatible data are with a null hypothesis.

Type I errors are false positives; Type II errors are false negatives.

Statistical significance may not mean practical importance.

Includes ANOVA, non-parametrics, bootstrapping, and maximum likelihood estimation.

LOTUS lets us find expectations of functions of random variables.

MGFs simplify calculations and uniquely determine distributions.

Markov Chains model systems that transition from one state to another, where the next state depends only on the current state.

Bayesian probability treats uncertainty as belief, not frequency.

Use Bayes’ Rule to revise probabilities as new data arrive.

Bayesian inference blends prior beliefs with data to form posterior beliefs.

MCMC, Gibbs Sampling, and modern tools like PyMC3 aid in complex Bayesian modeling.

Regression models relationships between a response and predictors.

OLS finds the best-fitting line by minimizing squared errors.

Extends regression to multiple predictors.

Residual analysis and other tests ensure assumptions are met.

Includes regularization, nonlinear, time series, and panel regressions.