Internal Links for Weekly Material

1_1.10

• Markov’s inequality : If $u(X)$ positive function $P[u(X)\ge c]\le \frac{E[u(X)]}{c}$

• Chebyshev’s inequality

$$\begin{array}{rl}P(|X-\mu |\ge k\sigma )& \le \frac{1}{k^2}\\ P(|X-\mu |<k\sigma )& \ge 1-\frac{1}{k^2}\end{array}$$

1_4.6

• Joint pdf of order statistics
• Marginal pdf of order statistics

$$\begin{array}{rl}{g}_1\left(y_1\right)& =\{\begin{array}{ll}n{\left[1-F\left(y_1\right)\right]}^{n-1}f\left(y_1\right),& a<y_1<b\\ 0,& \text{elsewhere}\end{array}\\ g_k\left(y_k\right)& =\{\begin{array}{ll}\frac{n!}{(n-k)!(k-1)!}{\left[1-F\left(y_k\right)\right]}^{n-k}{\left[F\left(y_k\right)\right]}^{k-1}f\left(y_k\right)& a<y_k<b\\ 0& \text{elsewhere}\end{array}\\ g_n\left(y_n\right)& =\{\begin{array}{ll}n{\left[F\left(y_n\right)\right]}^{n-1}f\left(y_n\right),& a<y_n<b\\ 0,& \text{elsewhere}\end{array}\end{array}$$

• CDF of order statistics

$$\begin{array}{rl}{F}_{Y_1}(x)& =1-[1-F_X(x){]}^n\\ F_{Y_k}(x)& =\sum _{j=k}^n\left(\genfrac{}{}{0ex}{}{n}{j}\right)[F_X(x){]}^j[1-F_X(x){]}^{n-j}\\ F_{Y_n}(x)& =[F_X(x){]}^n\end{array}$$

1_5.1

• Convergence in distribution : $X_n\stackrel{D}{\to }X$
• Def-degenerate-distribution

1_LIMITING DISTRIBUTIONS

2_5.2

• Convergence in probability : For all $\epsilon >0$
  • ${\lim }_{n\to \infty }P[|X_n-X\ge ϵ]=0$
  • ${\lim }_{n\to \infty }P[|X_n-X<ϵ]=1$
• Weak law of large numbers : ${\bar{X}}_n\stackrel{P}{\to }\mu$
• Weak law of large numbers for sample variance : $S_n^2\stackrel{P}{\to }{\sigma }^2$
• Strong law of large numbers
• Theorem 0 : $X_n\stackrel{P}{\to }c⟺X_n\stackrel{D}{\to }c$

2_5.3

• MGF Technique : ${\lim }_{n\to \infty }{M}_n(x)=M(x)⟹X_n\stackrel{D}{\to }X$

3_5.4

• Central limit theorem :
  • $Y_n=\frac{\sqrt{n}({\bar{X}}_n-\mu {)}^2}{\sigma }⟹Y_n\stackrel{D}{\to }N(0,1)$
  • $\sqrt{n}({\bar{X}}_n-\mu {)}^2⟹Y_n\stackrel{D}{\to }N(0,{\sigma }^2)$

3_5.5

• Theorem 1 : $U_n\stackrel{P}{\to }c⟹\frac{U_n}{c}\stackrel{P}{\to }1$
• Theorem 2 : $U_n\stackrel{P}{\to }c⟹\sqrt{U_n}\stackrel{P}{\to }\sqrt{c}$
• Theorem 2.5 : $U_n\stackrel{P}{\to }c\wedge V_n\stackrel{P}{\to }d⟹U_n{V}_n\stackrel{P}{\to }cd,\text{also,}\frac{U_n}{V_n}\stackrel{P}{\to }\frac{c}{d}$
• Theorem 3 : $U_n\stackrel{D}{\to }F(u)\wedge V_n\stackrel{P}{\to }1⟹\frac{U_n}{V_n}\stackrel{P}{\to }F(u)$

3_6.1

• Likelihood function : $L(\theta )={\prod }_{i=1}^{n}f(x_i;\theta )$
• Unbiased estimator : $E(T)=\theta$
• Consistent estimator : $T_n\stackrel{P}{\to }\theta$

3_6.1a_1

• MLE on normal distribution with multiple parameters

3_6.1a_2

• Method of Moments : ${\mu }_k^{\prime }=m_k^{\prime }$

3_7.2

• Sufficient statistic : $\frac{f}{f_{Y_1}}$ does not depend on $\theta$

3_7.2

• Neyman theorem : ${\prod }_{i=1}^{n}f(x_i;\theta )=k_1{k}_2$, where
  • $k_1,k_2$ nonnegative
  • $k_1$ function of $Y_1$
  • $k_2$ does not depend on $\theta$

4_7.4

• pmf: $E[u(Z)]=0$ for all $\theta$ implies $u(Z)=0$
• UMVUE: $Y$ complete sufficient, $\delta (Y)$ exist unbiased

4_7.5

• Regular exponential class $f(x;\theta )=\{\begin{array}{ll}\exp [p(\theta )K(x)+H(x)+q(\theta )]& x\in \mathcal{S}\\ 0& \text{elsewhere}\end{array}$
  • $\mathcal{S}$ independent on $\theta$
  • $p(\theta )$ nontrivial continuous
  • $X$ continuous → $K^{\prime }(x)\not\equiv 0$, $H(x)$ continuous
  • $X$ discrete → $K(x)$ nontrivial
• Theorem 7.5.2 $X$ regular exponential class, $Y_1=\sum K(X_i)$ $⟹$ $Y_1$ complete sufficient

5_7.6

• 7.6 Functions of Parameter

5_7.7

• Jointly Sufficient Statistic $\frac{{\prod }_{i=1}^{n}f(x;\mathbf{\theta })}{f_{\mathbf{Y}}(\mathbf{y};\mathbf{\theta })}$ does not depend on $\mathbf{\theta }$
• Regular Exponential Class on Random Vectors $$f(x;\mathbf{\theta })=\{\begin{array}{ll}\exp \left[{\sum }_{j=1}^m{p}_j(\mathbf{\theta })K_j(x)+H(x)+q({\theta }_1,\dots ,{\theta }_m)\right]& \forall x\in \mathcal{S}\\ 0& \text{elsewhere}\end{array}$$
  1. $\mathcal{S}$ independent of $\mathbf{\theta }$
  2. Space $\Omega$ contains a nonempty, $m$-dimensional open rectangle
  3. $p_j(\mathbf{\theta })$ are all nontrivial, functionally independent, continuous functions of $\mathbf{\theta }$
  4. If $X$ continuous random variable, then
     1. $K_j^{\prime }(x)$ are all continuous for $a<x<b$, not homogeneous linear function of the others.
     2. $H(x)$ : Continuous function of $x\in \mathcal{S}$
  5. If $X$ discrete random variable, then
     1. $K_j^{\prime }(x)$ are all nontrivial functions of $x\in \mathcal{S}$, not homogeneous linear function of the others

6_8.2

• Fisher Information

$$\begin{array}{rl}I(\theta )& =E\left[S(X;\theta {)}^2\right]\\ & =E\left[{\left(\frac{\partial }{\partial \theta }\ln f(X;\theta )\right)}^2\right]\\ & =-E\left[\frac{{\partial }^2}{\partial {\theta }^2}\ln f(X;\theta )\right]\end{array}$$

7_8.2

• Rao-Cramer Lower Bound: $\mathrm{Var}(Y)\ge \boxed{\frac{[k^{\prime }(\theta ){]}^2}{nI(\theta )}}$
• Rao-Cramer Lower Bound for Unbiased Estimators
  • def-efficiency
• Efficient Estimator
  • $Y$ unbiased
  • $nI(\theta )\cdot \mathrm{Var}(Y)=1$

8_6.2

• 4.2 Confidence Intervals

8_6.3

• 4.2.1 Confidence Intervals for Difference in Means
• 4.2.2 Confidence Interval for Difference in Proportion

9_6.4

• Hypothesis: Null hypothesis, Alternative hypothesis
• Test
• Critical region / Rejection region
• Power : $P_{\theta }(\mathbf{X}\in C)$
• Power Function : ${\gamma }_C(\theta )=P_{\theta }(\mathbf{X}\in C),\theta \in {\omega }_1$
• Types of Statistical Hypotheses: Simple statistical hypothesis, Composite statistical hypothesis
  • Simple: $H_0:\theta =10$
  • Composite: $H_0:\theta \le 10$
• Significance level / Size : $\alpha ={\max }_{\theta \in {\omega }_0}P(\mathbf{X}\in C)$

11_9.1

• Best Critical Region, Best Test
• Neyman-Pearson Theorem : $C$ best critical region of size $\alpha$ for test $H_0:\theta ={\theta }^{\prime }$ vs $H_1:\theta ={\theta }^{\prime \prime }$
  • $k>0$
  • $\alpha =P_{H_0}[\mathbf{X}\in C]$
  • $\frac{L({\theta }^{\prime };\mathbf{x})}{L({\theta }^{\prime \prime };\mathbf{x})}\le k,\forall \mathbf{x}\in C$

12_9.2

• UMPCR : $C$ from test simple $H_0$ vs composite $H_1$. If $C$ BCR from test simple $H_0$ vs all simple $H_1$, then $C$ UMPCR
• UMPT: If $C$ UMPCR, then its test UMPT
• Monotone Likelihood Ration (mlr) : If $\frac{L({\theta }_1,\mathbf{x})}{L({\theta }_2,\mathbf{x})},\forall {\theta }_1<{\theta }_2$ monotonic function of $y=u(\mathbf{x})$, then $L(\theta ,\mathbf{x})$ has MLR in $y=u(\mathbf{x})$

14_9.3

Also,

• Discrete Distributions
• Continuous Distributions