5.2 Convergence in Distribution

<< 5.1 Convergence in Probability.md | 5.3 Central Limit Theorem.md >>

Definition 5.2.1: Converges in distribution

Let

• $\{X_n\}$ : Sequence of random variables, with
  • $F_{X_n}$ : cdf of $X_n$
• $X$ : Random variable
  • $F_X$ : cdf of $X$
• $C(F_X)$ denote the set of all points where $F_X$ is continuous

If

$$\underset{x\to \infty }{\lim }{F}_{X_n}(x)=F_X(x),\forall x\in C(F_X)$$

Then

• We say $X_n$ converges in distribution to $X$
• We write $X_n\stackrel{D}{\to }X$

Link to original

Remark 5.2.2: Stirling’s formula

In advanced calculus, the following approximation is derived:

$$\Gamma (k+1)\approx \sqrt{2\pi }{k}^{k+1/2}{e}^{-k}$$

Theorem 5.2.1

$X_n\stackrel{P}{\to }X⟹X_n\stackrel{D}{\to }X$

Theorem 5.2.2

$X_n\stackrel{D}{\to }b⟺X_n\stackrel{P}{\to }b$

Note

Although in

• Hogg, R. V., McKean, J. W., & Craig, A. T. (2019). Introduction to Mathematical Statistics (8th ed.). Pearson.

Link to original

it is said stated that $X_n\stackrel{D}{\to }b⟹X_n\stackrel{P}{\to }b$, the converse is actually true. So we use $⟺$ instead.

Theorem 5.2.3

Suppose

• $X_n\stackrel{D}{\to }X$
• $Y_n\stackrel{P}{\to }0$

Then

$$X_n+Y_n\stackrel{D}{\to }X$$

Theorem 5.2.4

Suppose

• $X_n\stackrel{D}{\to }X$
• $g:\mathcal{S}\to X$ is continuous

Then

$$g(X_n)\stackrel{D}{\to }g(X)$$

Theorem 5.2.5: Slutsky’s theorem

Let

• $X_n,X,A_n,B_n$ : Random variables
• $a,b$ : Constants

If

• $X_n\stackrel{D}{\to }X$
• $X_n\stackrel{P}{\to }a$
• $B_n\stackrel{P}{\to }b$

Then

$$A_n+B_n{X}_n\stackrel{D}{\to }a+bX$$

Definition 5.2.2: Bounded in probability

Let

• $\{X_n\}$ : Sequence of Random variables

If

$$\forall ϵ>0,\exists B_{ϵ}>0,N_{ϵ}\in \mathbb{N}\ni n\ge N_{\epsilon }⟹P[|X_n|\le B_{\epsilon }]\ge 1-ϵ$$

Then we say $\{X_n\}$ is bounded in probability

Link to original

Theorem 5.2.6

Let

• $\{X_n\}$ : Sequence of random variables
• $X$ : Random variable

If

$$X_n\stackrel{D}{\to }X$$

Then $X_n$ is bounded in probability

Theorem 5.2.7

Let $\{X_n\},\{Y_n\}$ : Sequences of random variables

Suppose

• $\{X_n\}$ bounded in probability
• $Y_n\stackrel{P}{\to }0$

Then

$$X_n{Y}_n\stackrel{P}{\to }0$$

TODO: Make sense of 5.2.8, 5.2.9

Theorem 5.2.8

Let $\{Y_n\}$ : Sequence of random variables

Suppose

• $\{Y_n\}$ bounded in probability
• $X_n=o_p(Y_n)$

Then $X_n\stackrel{P}{\to }0$

Theorem 5.2.9

Let $\{X_n\}$ : Sequence of random variables

Suppose

• $\sqrt{n}(X_n-\theta )\stackrel{D}{\to }N(0,{\sigma }^2)$
• $g(x)$ is differentiable at $\theta$
• $g^{\prime }(\theta )\ne 0$

Then

$$\sqrt{n}[g(X_n)-g(\theta )]\stackrel{D}{\to }N(0,{\sigma }^2(g^{\prime }(\theta ){)}^2)$$

Theorem 5.2.10: MGF technique

Let

• $\{X_n\}$ : Sequence of random variables, with
  • mgf $M_{X_n}(t)$ which exists for $-h<t<h$ for all $n$
• $X$ : Random variable, with
  • mgf $M(t)$ which exists for $|t|\le h_1\le h$

If

$$\underset{n\to \infty }{\lim }{M}_{X_n}(t)=M(t),\forall |t|\le h_1$$

Then

$$X_n\stackrel{D}{\to }X$$

Note

The MGF uniquely determines a distribution (when it exists in a neighborhood of zero). So if the MGFs converge, the underlying distributions must also converge.

Theorem 0

Let

• $\{X_n\}$ : Sequence of random variables
• $c$ : Constant

Then

$$X_n\stackrel{P}{\to }c⟺X_n\stackrel{D}{\to }c$$

¹

Note

For convergence to a constant $c$, convergence in probability and convergence in distribution are equivalent.

Theorem 1

Let

• $U_n$ : Random variable
• $F_n(u)$ : cdf of $U_n$
• $c\ne 0$

Then $U_n\stackrel{P}{\to }c⟹\frac{U_n}{c}\stackrel{P}{\to }1$

Theorem 2

Let

• $U_n$ : Random variable
• $F_n(u)$ : cdf of $U_n$
• $c\in \mathbb{P}$

If

• $U_n\stackrel{P}{\to }c$
• $\mathrm{Pr}(U_n<0)=0,\forall n=1,2,\dots$

Then

$$\sqrt{U_n}\stackrel{P}{\to }\sqrt{c}$$

Theorem 2.5

Let

• $U_n,V_n$ : Sequences of random variables
• $c,d>0$ : Constants

If

• $U_n\stackrel{P}{\to }c$
• $V_n\stackrel{P}{\to }d$

Then

• $U_n{V}_n\stackrel{P}{\to }cd$
• $\frac{U_n}{V_n}\stackrel{P}{\to }\frac{c}{d},d\ne 0$

Theorem 3

Let

• $U_n$ : Random variable
• $V_n$ : Random variable
• $W_n=U_n/V_n$

If

• $U_n\stackrel{D}{\to }F(u)$
• $V_n\stackrel{P}{\to }1$

Then

$$W_n\stackrel{D}{\to }F(w)$$

Proposition 5.2.16

If ${\lim }_{n\to \infty }\varphi (n)=0$

Then

$$\begin{array}{rl}\underset{n\to \infty }{\lim }{\left[1+\frac{b}{n}+\frac{\varphi (n)}{n}\right]}^{cn}& =\underset{n\to \infty }{\lim }{\left(1+\frac{b}{n}\right)}^{cn}\\ & =e^{bc}\end{array}$$

Exercise

Exercise 5.1 of Hogg & Craig 5th ed.

Let ${\bar{X}}_n$ denote the mean of a random sample of size $n$ from a distribution that is $N(\mu ,{\sigma }^2)$. Find the limiting distribution of ${\bar{X}}_n$.

Based on Theorem 5.1.1: Weak law of large numbers,

$${\bar{X}}_n\stackrel{P}{\to }\mu$$

Consequently, the limiting distribution of ${\bar{X}}_n$ is $\mu$.

Hogg & Craig 5th ed. 5.7.

Let

• $Y_n$ : Sequence of random variable, with
  • $Y_n\sim b(n,p)$ (check here)

Prove that $1-Y_n/n$ converges in probability to $1-p$

Answer

Karena $Y_n\sim b(n,p)$, berarti $E[Y_n]=np$ dan $\mathrm{Var}(Y_n)=np(1-p)$. Misalkan ${\bar{Y}}_n=\frac{Y_n}{n}$. Maka,

• $E[\overline{Y_n}]=\frac{E[Y_n]}{n}=\frac{np}{n}=p$
• $\text{Var}(\overline{Y_n})=\frac{\text{Var}(Y_n)}{n^2}=\frac{np(1-p)}{n^2}=\frac{p(1-p)}{n}$

Misalkan $k=\frac{ϵ}{\sigma }$. Berdasarkan teorema Chebyshev,

$$\begin{array}{rl}P\left(\left|{\bar{Y}}_n-E[X]\right|>k\sigma \right)& \le \frac{1}{k^2}\\ P\left(\left|\frac{Y_n}{n}-p\right|>ϵ\right)& \le \frac{{\sigma }^2}{{ϵ}^2}\\ & \le \frac{\mathrm{Var}({\bar{Y}}_n)}{{ϵ}^2}=\frac{p(1-p)}{n{ϵ}^2}\\ \underset{n\to \infty }{\lim }P\left(\left|\frac{Y_n}{n}-p\right|>ϵ\right)& \le \underset{n\to \infty }{\lim }\frac{p(1-p)}{n{ϵ}^2}=0\end{array}$$

Sehingga diperoleh

$$\underset{n\to \infty }{\lim }P\left(\left|1-\frac{Y_n}{n}-(1-p)\right|>ϵ\right)=0$$

Berdasarkan definisi konvergen dalam probabilitas, terbukti bahwa $1-\frac{Y_n}{n}\stackrel{P}{\to }1-p$

Hogg & Craig 5th ed. 5.10.

Let

• $Y_n$ : $n$-th Order statistic from random sample of size $n$, with
  • $Y_n\sim U(0,\theta )$

Prove that $Z_n=\sqrt{Y_n}$ converges in probability to $\sqrt{\theta }$

Answer

Dari Example 1 Section 5.1

$Y_n$ converges in distribution to a random variable that has a degenerate distribution at the point $y=\theta$.

Artinya, $Y_n\stackrel{D}{\to }\theta$. Berdasarkan teorema 5.2.2, diperoleh $Y_n\stackrel{P}{\to }\theta$. Misalkan $Z_n=g(Y_n)=\sqrt{Y_n}$. Berdasarkan teorema 5.1.4, diperoleh $Z_n\stackrel{P}{\to }\theta$, sehingga terbukti bahwa $Z_n=\sqrt{Y_n}$ konvergen dalam probabilitas ke $\sqrt{\theta }$

Hogg & Craig 5th ed. 5.12.

Let

• $Z_n$ : Sequence of random variables, with $Z_n\sim {\chi }^2(n)$
• $W_n=Z_n/n^2$

Find the limiting distribution of $W_n$

Answer

Diketahui mgf dari distribusi Chi-square adalah $M_{Z_n}(t)=(1-2t{)}^{-n/2}$, untuk $t<\frac{1}{2}$. Karena $W_n=\frac{Z_n}{n^2}$, mgf dari $W_n$ dapat diperoleh dengan

$$\begin{array}{rl}{M}_{W_t}& =E[e^{t{W}_n}]\\ & =E\left[e^{\frac{t}{n^2}{Z}_n}\right]\\ & =M_{Z_n}\left(\frac{t}{n^2}\right)\\ & ={\left(1-\frac{2t}{n^2}\right)}^{-n/2},\text{Untuk }\frac{t}{n^2}<\frac{1}{2}\end{array}$$

Karena ${\lim }_{n\to \infty }{\left(1+\frac{b}{n}\right)}^{cn}=e^{bc}$, dengan memisalkan $b=-2t$, $c=1$, dan $n=n^2$, dapat diperoleh

$$\begin{array}{rl}{M}_{W_n}(t)& ={\left(1-\frac{2t}{n^2}\right)}^{-n/2}\\ & ={\left[{\left(1-\frac{2t}{n^2}\right)}^{n^2}\right]}^{-1/(2n)}\\ \underset{n\to \infty }{\lim }{M}_{W_n}(t)& =\underset{n\to \infty }{\lim }{\left(e^{-2t}\right)}^{-1/(2n)}\\ & =\underset{n\to \infty }{\lim }{e}^{t/n}\\ & =e^0\\ & =1\end{array}$$

Misalkan $X$ variabel acak dengan mgf $M_X(t)$. Andaikan $X$ berdistribusi degenerate ke $0$, yang berakibat $M_X(t)=E\left[e^{tX}\right]=e^{t\cdot 0}=1$. Karena ${\lim }_{n\to \infty }{M}_{W_n}(t)=M_X(t)$, berdasarkan teorema 5.2.10, diperoleh $W_n\stackrel{D}{\to }X$. Terbukti bahwa limiting distribution dari $W_n$ adalah $X$.

Hogg & Craig 5th ed. 5.13.

Let

• $X$ : Random variable, with
  • $X\sim {\chi }^2(50)$

Approximate $\mathrm{Pr}(40<X<60)$

Answer

Hogg & Craig 5th ed. 5.15.

Let

• $Z_n$ : Sequence of random variable, with $Z_n\sim P(n)$
• $Y_n=(Z_n-n)/\sqrt{n}$

Show that the limiting distribution of $Y_n$ is normal with mean zero and variance $1$.

Answer

Diketahui mgf dari distribusi $P(n)$ adalah $M_{Z_n}=e^{n(e^t-1)}$. Dapat diperoleh mgf dari $M_{Y_n}$:

$$\begin{array}{rl}{M}_{Y_n}(t)& =E\left[e^{t{Y}_n}\right]\\ & =E\left[e^{t\cdot \frac{Z_n-n}{\sqrt{n}}}\right]\\ & =e^{-t\sqrt{n}}E\left[e^{t/\sqrt{n}{Z}_n}\right]\\ & =e^{-t\sqrt{n}}{M}_{Z_n}\left(\frac{t}{\sqrt{n}}\right)\\ & =e^{-t\sqrt{n}}{e}^{n(e^{t/\sqrt{n}}-1)}\end{array}$$

Menggunakan Maclaurun series pada $f(x)=e^x$, perluaskan $e^{t/\sqrt{n}}$ menjadi

$$e^{t/\sqrt{n}}=1+\frac{t}{\sqrt{n}}+\frac{t^2}{2n}+o\left(\frac{1}{n}\right)=\sum _{n=0}^{\infty }\frac{(t/\sqrt{n}{)}^n}{n!}$$

Sehingga diperoleh

$$\begin{array}{rl}{M}_{Y_n}(t)& =e^{-t\sqrt{n}}{e}^{n(e^{t/\sqrt{n}}-1)}\\ & =e^{-t\sqrt{n}}{e}^{n\left(t/\sqrt{n}+t^2/2n+o(1/n)\right)}\\ & =e^{-t\sqrt{n}}{e}^{t\sqrt{n}+t^2/2+o(1)}\\ & =e^{t^2/2+o(1)}\\ \underset{n\to \infty }{\lim }{M}_{Y_n}(t)& =\underset{n\to \infty }{\lim }{e}^{t^2/2+o(1)}\\ & =e^{t^2/2}\end{array}$$

Misalkan $X$ variabel acak dengan berdistribusi $N(0,1)$. Artinya, $M_X(t)=e^{t^2/2}$. Berdasarkan teorema 5.2.10, karena ${\lim }_{n\to \infty }{M}_{Y_n}(t)=M_X(t)$, maka $Y_n\stackrel{D}{\to }X$, sehingga $Y_n\stackrel{D}{\to }N(0,1)$. Terbukti bahwa limiting distribution dari $Y_n$ adalah distribusi normal dengan mean $0$ dan variansi $1$.

Example 1

Example 2

Footnotes

1. Taken from Theorem 1 in Section 5.2: Convergence in Probability of Hogg & Craig 5th ed. ↩