7.3 Properties of a Sufficient Statistic

<< 7.2 A Sufficient Statistic for a Parameter.md | 7.4 Completeness and Uniqueness.md >>

Theorem 7.3.1: Rao-Blackwell

Let

• $X_1,\dots X_n$ : Random sample, with
  • pdf/pmf $f(x;\theta )$, $\theta \in \Omega$
• $Y_1=u_1(X_1,\dots ,X_n)$ : Sufficient statistic for $\theta$
• $Y_2=u_2(X_1,\dots ,X_n)$ : Unbiased estimator for $\theta$

If $E(Y_2|y_1)=\phi (y_1)$ defines a statistic $\phi (Y_1)$

Then

• $\phi (Y_1)$ is an unbiased estimator of $\theta$
• $\mathrm{Var}[\phi (Y_1)]\le \mathrm{Var}(Y_2)$

This theorem states “function $\phi (y_1)=E(Y_2|y_1)$ of the sufficient statistic $Y_1$ is an unbiased estimator of $\theta$ having a smaller variance than that of the unbiased estimator $Y_2$ of $\theta$“.

In simpler terms, given

• $Y_1$ : Sufficient statistic for $\theta$
• $Y_2$ : Unbiased statistic for $\theta$

A function of $Y_1$ (usually denoted as $\phi (Y_1)$) is a better (lower/equal variance) unbiased estimator than $Y_2$.

Theorem 7.3.2

Let

• $X_1,\dots ,X_n$ : Random sample, with
  • pdf/pmf $f(x;\theta )$, $\theta \in \Omega$
• $Y_1=u_1(X_1,\dots ,X_n)$ : Sufficient statistic for $\theta$
• $\hat{\theta }$ : Maximum likelihood estimator of $\theta$

If

• $Y_1$ exists
• $\hat{\theta }$ exists uniquely

Then $\hat{\theta }$ is a function of $Y_1$