Eigenvalue

Definition

Let

• $A$ : $n\times n$ square matrix
• $\mathbf{x}$ : nonzero vector in $R^n$¹

If for some scalar $\lambda$

$$A\mathbf{x}=\lambda \mathbf{x}$$

Then

• $\lambda$ is called an eigenvalue of $A$
• $\mathbf{x}$ is called an eigenvector corresponding to $\lambda$

Procedure: Finding eigenvalue

Let $A$ be matrix of interest.

Based on Proof 1, the procedure is simply to:

1. Solve for $\lambda I$ : $\det (A-\lambda I)=0$²

Example: Finding eigenvalue

Let

$$A=\left[\begin{array}{cc}3& 1\\ 1& 3\end{array}\right]$$

The eigenvalue of $A$ is:

$$\begin{array}{rl}\det (A-\lambda I)& =\det \left[\begin{array}{cc}3-\lambda & 1\\ 1& 3-\lambda \end{array}\right]=0\\ (3-\lambda )(3-\lambda )-(1)(1)& =0\\ 9-6\lambda +{\lambda }^2-1& =0\\ {\lambda }^2-6\lambda +8& =0\\ (\lambda -4)(\lambda -2)& =0\\ & \end{array}$$

The last equation above is true when $\lambda =4$ and $\lambda =2$. Thus,

$$\lambda I=\left[\begin{array}{cc}4& 0\\ 0& 2\end{array}\right]$$

Note

The order of eigenvalues doesn’t matter, but in some cases like Singular Value Decomposition, the eigenvalues are sorted in descending order in the columns of $\lambda I$.

Procedure: Finding eigenvector

Let $A$ be matrix of interest.

1. Find eigenvalue(s) of $A$
2. Solve for $\mathbf{x}$ : $(A-\lambda I)\mathbf{x}=0$ for each eigenvalue

Example: Finding eigenvector

For ${\lambda }_1=4$:

$$\begin{array}{rl}(A-4I)\mathbf{x}& =0\\ \left[\begin{array}{cc}-1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]& =\left[\begin{array}{c}0\\ 0\end{array}\right]\\ & \\ -x_1+x_2& =0\\ x_2& =x_1\\ & \\ {\mathbf{v}}_1& =\left[\begin{array}{c}1\\ 1\end{array}\right]\end{array}$$

Note

Since $x_2=x_1$, we have infinitely many solutions. Then \begin{align} \mathbf{x} & = \begin{bmatrix} x_{1} \\ x_{2} \end{bmatrix} \\ & = \begin{bmatrix} x_{1} \\ x_{1} \end{bmatrix} \\ & = x_1\begin{bmatrix} 1 \\ 1 \end{bmatrix} \end{align}

Setting $x_1=1$ gives the eigenvector $\mathbf{x}={\mathbf{v}}_1=\left[\begin{array}{c}1\\ 1\end{array}\right]$. Any other arbitrary $x_1$ is also valid.

For ${\lambda }_2=2$:

$$\begin{array}{rl}(A-2I)\mathbf{x}& =0\\ \left[\begin{array}{cc}1& 1\\ 1& 1\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]& =\left[\begin{array}{c}0\\ 0\end{array}\right]\\ & \\ x_1+x_2& =0\\ x_2& =-x_1\\ & \\ {\mathbf{v}}_2& =\left[\begin{array}{c}1\\ -1\end{array}\right]\end{array}$$

Proof 1

$$\begin{array}{rl}A\mathbf{x}& =\lambda \mathbf{x}\\ A\mathbf{x}-\lambda \mathbf{x}& =0\\ A\mathbf{x}-(\lambda I)\mathbf{x}& =0\\ (A-\lambda I)\mathbf{x}& =0\end{array}$$

Notice that this system always has the trivial solution $\mathbf{x}=0$, but $\mathbf{x}$ is by definition, a nonzero vector.

Based on (a) and (g) of matrix equivalency statements, the negation of the biconditional statement states ”$\det (B)=0⟺B\mathbf{x}=0$ has nontrivial solutions”.

Therefore, for non-trivial solutions for $\mathbf{x}$ to exist, determinant of $(A-\lambda I)$ must be $0$, i.e.,

$$\det (A-\lambda I)=0$$

Footnotes

1. Vector Space ↩

2. Determinant of Matrices ↩