4.2.1 Confidence Intervals for Difference in Means

<< 4.2 Confidence Intervals | 4.2.2 Confidence Interval for Difference in Proportion >>

Proof: Confidence interval for difference in mean, inequal variances

Let

• $X,Y$ : Random variable
• ${\mu }_1$ : Mean of $X$
• ${\mu }_2$ : Mean of $Y$
• $\Delta ={\mu }_1-{\mu }_2$
• ${\sigma }_1^2=\mathrm{Var}(X),{\sigma }_2^2=\mathrm{Var}(Y)$ exists

Let

• $X_1,\dots ,X_{n_1}$ : Random sample from $X$
• $Y_1,\dots ,Y_{n_2}$ : Random sample from $Y$
• $\bar{X}=\frac{1}{n}{\sum }_{i=1}^{n_1}{X}_i$ : Sample mean of $X_i$
• $\bar{Y}=\frac{1}{n}{\sum }_{i=1}^{n_1}{Y}_i$ : Sample mean of $Y_i$

Assume the random samples $X_i$ and $Y_i$ are independent.

Then $\bar{X}\sim N\left({\mu }_1,\frac{{\sigma }_1^2}{n_1}\right)$ and $\bar{Y}\sim N\left({\mu }_2,\frac{{\sigma }_2^2}{n_2}\right)$

Let

• $\hat{\Delta }=\bar{X}-\bar{Y}$

Then $E(\hat{\Delta })=E(\bar{X}-\bar{Y})=E(\bar{X})-E(\bar{Y})={\mu }_1-{\mu }_2$, thus $\hat{\Delta }$ is an unbiased estimator of $\Delta$. Additionally, $\mathrm{Var}(\hat{\Delta })=\mathrm{Var}(\bar{X}-\bar{Y})=\mathrm{Var}(\bar{X})+\mathrm{Var}(\bar{Y})=\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}$

Let

• $S_1^2=\frac{1}{n_1-1}{\sum }_{i=1}^{n_1}(X_i-\bar{X}{)}^2$ : Sample variance of $X_i$
• $S_2^2=\frac{1}{n_2-1}{\sum }_{i=1}^{n_2}(Y_i-\bar{Y}{)}^2$ : Sample variance of $Y_i$

Then $S_1^2\sim N\left({\sigma }^2,\frac{2{\sigma }^4}{n-1}\right)$ and $S_2^2\sim N\left({\sigma }_2^2,\frac{2{\sigma }_2^4}{n_2-1}\right)$

By independence of the random samples, we may obtain the distribution for $\hat{\Delta }$ by standartization:

$$\begin{array}{rl}\bar{X}-\bar{Y}=\hat{\Delta }& \sim N\left({\mu }_1-{\mu }_2,\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}\right)\\ ⟺\frac{\hat{\Delta }-({\mu }_1-{\mu }_2)}{\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}}& \sim N(0,1)\\ ⟺\frac{\hat{\Delta }-\Delta }{\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}}& \sim N(0,1)\end{array}$$

Thus similar to Example 4.2.1, its $(1-\alpha )100\%$ confidence interfal for $\Delta$ is

$$\begin{array}{rl}1-\alpha & =P\left(z_{\alpha /2}<\frac{\hat{\Delta }-\Delta }{\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}}<z_{\alpha /2}\right)\\ & =P\left(z_{\alpha /2}\sqrt{\frac{{\sigma }_1^2}{n_1}}+\frac{{\sigma }_2^2}{n_2}<\hat{\Delta }-\Delta <-z_{\alpha /2}\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}\right)\\ & =P\left(-\hat{\Delta }+z_{\alpha /2}\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}<-\Delta <-\hat{\Delta }-z_{\alpha /2}\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}\right)\\ & =P\left(\hat{\Delta }-z_{\alpha /2}\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}<\Delta <\hat{\Delta }+z_{\alpha /2}\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}\right)\\ & =P\left(\hat{\Delta }-z_{\alpha /2}\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}<{\mu }_1-{\mu }_2<\hat{\Delta }+z_{\alpha /2}\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}\right)\end{array}$$

By estimating the variance by the sample variances, this leads to the approximate $(1-\alpha )100\%$ confidence interval for $\Delta ={\mu }_1-{\mu }_2$ given by $\left((\bar{x}-\bar{y})-z_{\alpha /2}\sqrt{\frac{s_1^2}{n_1}+\frac{s_1^2}{n_1}},(\bar{x}-\bar{y})+z_{\alpha /2}\sqrt{\frac{s_1^2}{n_1}+\frac{s_1^2}{n_1}}\right)$

Proof: Confidence interval for difference in mean, equal variances

Continuing from the previous example, assume ${\sigma }_1^2={\sigma }_2^2={\sigma }^2$. Thus the distributions can differ only in location, i.e., a location model

Assume $X\sim N({\mu }_1,{\sigma }^2)$ and $Y\sim N({\mu }_2,{\sigma }^2)$.

Continuing from the previous example

$$\begin{array}{rl}\frac{\hat{\Delta }-\Delta }{\sqrt{\frac{{\sigma }_1^2}{n_1}+\frac{{\sigma }_2^2}{n_2}}}& \sim N(0,1)\\ \frac{(\bar{X}-\bar{Y})-({\mu }_1-{\mu }_2)}{\sqrt{{\sigma }^2\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}}& \sim N(0,1)\\ \frac{(\bar{X}-\bar{Y})-({\mu }_1-{\mu }_2)}{\sigma \sqrt{\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}}& \sim N(0,1)\end{array}\text{\hspace{1em}(1)}$$

Let $S_p^2=\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}$

Recall $E(S_1)={\sigma }_1^2,E(S_2)={\sigma }_2^2,{\sigma }_1^2={\sigma }_2^2={\sigma }^2$. As a result,

$$\begin{array}{rl}E(S_p^2)& =E\left[\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}\right]\\ & =\frac{E\left[(n_1-1)S_1^2+(n_2-1)S_2^2\right]}{n_1+n_2-2}\\ & =\frac{E\left[(n_1-1)S_1^2\right]+E\left[(n_2-1)S_2^2\right]}{n_1+n_2-2}\\ & =\frac{(n_1-1)E(S_1^2)+(n_2-1)E(S_2^2)}{n_1+n_2-2}\\ & =\frac{(n_1-1){\sigma }_1^2+(n_2-1){\sigma }_2^2}{n_1+n_2-2}\\ & =\frac{{\sigma }^2[(n_1-1)+(n_2-1)]}{n_1+n_2-2}\\ & =\frac{{\sigma }^2(n_1+n_2-2)}{n_1+n_2-2}\\ & ={\sigma }^2\end{array}$$

Therefore, $S_p^2$ is an unbiased estimator of ${\sigma }^2$. We call it the pooled estimator of ${\sigma }^2$.

By Normal Distribution Relationships, $(n_1-1)S_1^2/{\sigma }^2\sim {\chi }^2(n_1-1)$ and $(n_2-1)S_2^2/{\sigma }^2\sim {\chi }^2(n_2-1)$. Also, $S_1^2$ and $S_2^2$ are independent. Therefore, by Corollary 3.3.1,

$$\begin{array}{rl}\frac{(n_1-1)S_1^2}{{\sigma }^2}+\frac{(n_2-1)S_2^2}{{\sigma }^2}& \sim {\chi }^2(n_1-1+n_2-2)\\ \frac{S_p^2(n-2)}{{\sigma }^2}& \sim {\chi }^2(n-2)\end{array}$$

Finally, because $S_1^2$ and $S_2^2$ is independent of $\bar{X}$ and $\bar{Y}$ respectively, and the random samples are independent of each other, it follows that $S_p^2$ is independent of expression $(1)$.

Thus by 3.6.1 The t-distribution we may construct a random variable with t-distribution:

$$\begin{array}{rl}T& =\frac{[(\bar{X}-\bar{Y})-({\mu }_1-{\mu }_2)]/\sigma \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}{\sqrt{(n-2)S_p^2/(n-2){\sigma }^2}}\\ & =\frac{(\bar{X}-\bar{Y})-({\mu }_1-{\mu }_2)}{S_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}\sim t_{n-2}\end{array}$$

The confidence interval may then be found:

$$\begin{array}{rl}1-\alpha & =P\left(t_{n-2}<\frac{(\bar{X}-\bar{Y})-({\mu }_1-{\mu }_2)}{S_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}<t_{n_2}\right)\\ & =P\left(t_{n-2}{S}_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}<(\bar{X}-\bar{Y})-({\mu }_1-{\mu }_2)<t_{n_2}{S}_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}\right)\\ & =P\left((\bar{x}-\bar{y})-t_{n-2}{S}_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}<{\mu }_1-{\mu }_2<(\bar{x}-\bar{y})+t_{n_2}{S}_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}\right)\end{array}$$

From the last result, we can see that the following interval is an exact $(1-\alpha )100\%$ confidence interval for $\Delta ={\mu }_1-{\mu }_2$

$$\left((\bar{x}-\bar{y})-t_{(\alpha /2,n-2)}{s}_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}},(\bar{x}-\bar{y})+t_{(\alpha /2,n-2)}{s}_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}},\right)$$