Simple Linear Model

General form

$$\begin{array}{c}Y={\beta }_0+{\beta }_{1}X+{ϵ}_1,ϵ\sim NIID(0,{\sigma }^2)\end{array}$$

• $Y$: Response or dependent variable
• $X$: Predictor, explanatory, or independent variable
• ${\beta }_0$: Intercept
• ${\beta }_1$: Slope
• $ϵ$: Error component

We denote our sample estimations for the $i$-th observation using $y_i={\beta }_0+{\beta }_1{x}_i+{ϵ}_i$.

Note

The predictor $X$ in linear model can be a quadratic, log, fractions, etc. As long as it can be substituted to obtain the form above, it’s still considered as a linear model

e.g., $Y={\beta }_0+\beta \log (x)+ϵ$. Substitute with $x^{\ast }=\log (x)$, then you can see that it’s a linear model.

Matrix form

$$\begin{array}{rl}\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_n\end{array}\right]& =\left[\begin{array}{c}{\beta }_0+{\beta }_1{x}_1+{ϵ}_1\\ {\beta }_0+{\beta }_1{x}_2+{ϵ}_2\\ ⋮\\ {\beta }_0+{\beta }_1{x}_n+{ϵ}_n\end{array}\right]\\ & =\left[\begin{array}{cc}1& x_1\\ 1& x_2\\ ⋮& ⋮\\ 1& x_n\end{array}\right]\left[\begin{array}{c}{\beta }_0\\ {\beta }_1\end{array}\right]+\left[\begin{array}{c}{ϵ}_1\\ {ϵ}_2\\ ⋮\\ {ϵ}_n\end{array}\right]\\ \mathbf{y}& =\mathbf{x}\mathbf{\beta }+\mathbf{ϵ};\mathbf{ϵ}\sim MVN(0,\mathbf{\Sigma })\end{array}$$

Note

This matrix form can be extended for linear model with multiple predictor