Introduction to Mathematical Statistics

• italic: Definition
• Bold: Theorem/Corollary

If you’re looking for summaries, you may be interested in Cheatsheets or Materi Statmat 2

Content by Hogg & Craig, 8th ed

Chapter 1: Probability and distributions

• 1.1 Introduction
  • Random Experiment
  • Sample Space
  • Events
• 1.2 Sets : See sets
• 1.3 The Probability Set Function
  • Probability Set Function
  • Property of Probability Set Function
  • Limit Theorem for Monotone Events
  • Boole’s Inequality
  • Equilikely Case
• 1.4 Conditional Probability and Independence
  • Conditional Probability
  • Independent Events
  • Bayes Theorem
• 1.5 Random Variables
  • Random Variable
  • Cumulative Distribution Function (cdf)
  • Equal in Distribution
• 1.6 Discrete Random Variables
  • Discrete Random Variable
  • Probability Mass Function (pmf)
  • Support of Discrete Random Variable
• 1.7 Continuous Random Variables
  • Continuous Random Variable
  • Probability Density Function (pdf)
  • Support of Continuous Random Variable
  • Quantile
  • Finding the pdf of a Transformation
• 1.8 Expectation of Random Variable
  • Expectation
• 1.9 Some Special Expectations
  • Mean
  • Variance
  • Moments
  • Characteristic Function
  • Moment Generating Function (mgf)
• 1.10 Important Inequalities
  • Existence of Lower Order Moments
  • Markov’s Inequality
  • Chebyshev’s Inequality
  • Convex Function
  • Jensen’s Inequality

Chapter 2: Multivariate distributions

• 2.1 Distributions of Two Random Variables
  • Random Vector TODO
• 2.2 Transformations: Bivariate Random Variables
• 2.3 Conditional Distributions and Expectations
  • Conditional pmf
  • Conditional pdf
• 2.6 Extension to Several Random Variables
  • Space TODO

Chapter 3: Some special distributions

• 3.1 The Binomial and Related Distributions
  • 3.1.1 Negative Binomial and Geometric Distributions
  • 3.1.2 Multinomial Distribution
  • 3.1.3 Hypergeometric Distribution
• 3.2 The Poisson Distribution
• 3.3 The Gamma, Chi-square, and Beta Distributions
  • 3.3.1 The Chi-square Distribution
  • 3.3.2 The Beta Distribution
• 3.4 The Normal Distribution
  • 3.4.1 *Contaimnated Normals
• 3.5 The Multivariate Normal Distribution
  • 3.5.1 Bivariate Normal Distribution
  • 3.5.2 *Multivariate Normal Distribution, General Case
  • 3.5.3 *Applications
• 3.6 t- and F-Distributions
  • 3.6.1 The t-distribution
  • 3.6.2 The F-distribution
  • 3.6.3 Student’s Theorem
• 3.7 *Mixture Distributions

Chapter 4: Some elementary statistical inferences

• 4.1 Sampling and Statistics
  • Random Sample
  • Statistic
  • Estimator
  • Unbiased Estimator
  • Likelihood Function
• 4.2 Confidence Intervals
  • Confidence Interval
  • 4.2.1 Confidence Intervals for Difference in Means
  • 4.2.2 Confidence Interval for Difference in Proportion
• 4.4 Order Statistics
  • Order Statistics
  • Joint pdf of Order Statistics
  • Marginal pdf of Order Statistics
  • cdf of Order Statistics
• 4.5 Introduction to Hypothesis Testing
  • Hypothesis
  • Test
  • Test Error Types
  • Size of Critical Region
  • Power of a Test
  • Power Function
  • Types of Statistical Hypotheses

Chapter 5: Concistency and limiting distributions

• 5.1 Convergence in Probability
  • Convergence in Probability
  • Properties of Convergence in Probability
  • Weak Law of Large Numbers
  • Consistent Estimator
  • Almost Sure Convergence
• 5.2 Convergence in Distribution
  • Convergence in Distribution
  • Bounded in Probability
• 5.3 Central Limit Theorem
  • Central Limit Theorem
• 5.4 Extensions to Multivariate Distributions
  • Multivariate Convergence in Probability
  • Multivariate Convergence in Distribution

Chapter 6: Maximum likelihood methods

• 6.1 Maximum Likelihood Estimation
  • Maximum Likelihood Estimator (mle)
• 6.2 Rao-Cramér Lower Bound and Efficiency
  • Score Function
  • Fisher Information
  • Efficient Estimator
  • Rao-Cramer Lower Bound
  • Regularity Conditions
• 6.3 Maximum Likelihood Tests

Chapter 7: Sufficiency

• 7.1 Measures of Quality Estimators
  • Minimum Variance Unbiased Estimator (MVUE)
• 7.2 A Sufficient Statistic for a Parameter
  • Sufficient Statistic
  • Neyman Theorem
• 7.3 Properties of a Sufficient Statistic
• 7.4 Completeness and Uniqueness
  • pmf
  • Complete Sufficient Statistic
  • Unique MVUE (UMVUE)
• 7.5 The Exponential Class of Distributions
  • Regular Exponential Class
• 7.6 Functions of Parameter TODO: insert examples
• 7.7 The Case of Several Parameters
  • Jointly Sufficient Statistic
  • Regular Exponential Class on Random Vectors

Chapter 8: Optimal Tests of Hypotheses

• 8.1 Most Powerful Tests
  • Best Critical Region
  • Neyman-Pearson Theorem
  • Unbiased Test
• 8.2 Uniformly Most Powerful Test
  • Uniformly Most Powerful Critical Region
  • Uniformly Most Powerful Test
  • Monotone Likelihood Ration (mlr)
• 8.3 Likelihood Ratio Tests
  • Likelihood Ratio Test
  • Central t-Distribution

Definition hierarchy

Probability Fundamentals

• Random Variables
  • Random Variable
    • Discrete Random Variable
    • Continuous Random Variable
    • Degenerate Distribution
• Expectation & Moments
  • Expectation
    • Mean
    • Variance
    • Moments
• Convergence
  • Convergence in Probability
  • Convergence in Distribution
  • Almost Sure Convergence

Statistical Inference

• Statistics
  • Statistic
    • Order Statistics
    • Sufficient Statistic : $\frac{f}{f_{Y_1}}$ does not depend on $\theta$
      • Complete Sufficient Statistic
      • Jointly Sufficient Statistic
• Estimation
  • Estimator
    • Unbiased estimator : $E(T)=\theta$
      • MVUE : $Y$ unbiased, lowest variance
      • UMVUE : $Y$ complete sufficient, $\delta (Y)$ unbiased
      • Efficient Estimator : Attains Rao-Cramér bound ($nI(\theta )\cdot \mathrm{Var}(Y)=1$)
    • Consistent Estimator : $T\stackrel{P}{\to }\theta$
  • Methods
    • Maximum Likelihood Estimation (MLE) / Maximum Likelihood Estimator : $\hat{\theta }=\mathrm{Argmax}L(\theta )$
    • Method of Moments : ${\mu }_k^{\prime }=m_k^{\prime }$
• Information Theory
  • Score Function
  • Fisher Information

Hypothesis Testing

• Test
  • Critical region
  • Best Critical Region (Best Test)
  • Uniformly Most Powerful (UMP) Critical Region
  • Unbiased Test

Distribution Families

• pmf
• Regular Exponential Class
  • Regular Exponential Class on Random Vectors

Miscellaneous

• Combination and Permutation

Cheatsheets

• Continuous Distributions
• Discrete Distributions
• Common Expectation and Variance Operations
• Common Distribution Equations
• Common Confidence Intervals

Exercises

• Exercises from 5th ed Book

References

• Hogg, R. V., McKean, J. W., & Craig, A. T. (2019). Introduction to Mathematical Statistics (8th ed.). Pearson.
• Hogg, R. V., & Craig, A. T. (1995). Introduction to mathematical statistics (5th ed.). Prentice Hall.

Author notes

• See also Materi Statmat 2