Positive Definite Matrix

Definition

Let $A$ : $n\times n$ symmetric matrix

Then $A$ is positive definite if for all nonzero vectors $\mathbf{x}\in {\mathbb{R}}^n$:

${\mathbf{x}}^{T}A\mathbf{x}>0$

Similarly:

• $A$ is positive semidefinite if ${\mathbf{x}}^{T}A\mathbf{x}\ge 0$ for all $\mathbf{x}\in {\mathbb{R}}^n$
• $A$ is negative definite if ${\mathbf{x}}^{T}A\mathbf{x}<0$ for all nonzero $\mathbf{x}\in {\mathbb{R}}^n$
• $A$ is negative semidefinite if ${\mathbf{x}}^{T}A\mathbf{x}\le 0$ for all $\mathbf{x}\in {\mathbb{R}}^n$
• $A$ is indefinite if ${\mathbf{x}}^{T}A\mathbf{x}$ takes both positive and negative values

Theorems

Let $A$ : $n\times n$ symmetric matrix

Eigenvalue Characterization Theorem

$A$ is positive definite if and only if all eigenvalues of $A$ are positive.

Similarly:

• $A$ is positive semidefinite if and only if all eigenvalues are nonnegative
• $A$ is negative definite if and only if all eigenvalues are negative
• $A$ is negative semidefinite if and only if all eigenvalues are nonpositive

Definiteness	All eigenvalues ${\lambda }_i$ satisfy
Positive	${\lambda }_i>0$
Positive semi	${\lambda }_i\ge 0$
Negative	${\lambda }_i<0$
Negative semi	${\lambda }_i\le 0$
Indefinite	Some ${\lambda }_i>0$ and some ${\lambda }_j<0$

Principal Minors Theorem (Sylvester’s Criterion)

$A$ is positive definite if and only if all leading principal minors are positive.

The $k$-th leading principal minor is: $\det \left[\begin{array}{ccc}{a}_{11}& \cdots & a_{1k}\\ ⋮& \ddots & ⋮\\ a_{k1}& \cdots & a_{kk}\end{array}\right]$

for $k=1,2,\dots ,n$.

Cholesky Decomposition Theorem

$A$ is positive definite if and only if there exists a unique lower triangular matrix $L$ with positive diagonal entries such that:

$A=L{L}^T$

This is called the Cholesky decomposition of $A$.

Invertibility Theorem

If $A$ is positive definite, then $A$ is invertible and $A^{-1}$ is also positive definite.

Quadratic Form Theorem

For any $n\times n$ matrix $B$, $B^{T}B$ is positive semidefinite.

If $B$ has full column rank, then $B^{T}B$ is positive definite.

Note

This is why $A^{T}A$ in Singular Value Decomposition always has nonnegative eigenvalues.

Examples

Verifying positive definiteness using the definition

Let $A=\left[\begin{array}{cc}2& 1\\ 1& 2\end{array}\right]$

For any nonzero $\mathbf{x}=\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]$:

$$\begin{array}{rl}{\mathbf{x}}^{T}A\mathbf{x}& =\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]\left[\begin{array}{cc}2& 1\\ 1& 2\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]\\ & =\left[\begin{array}{cc}{x}_1& x_2\end{array}\right]\left[\begin{array}{c}2x_1+x_2\\ x_1+2x_2\end{array}\right]\\ & =x_1(2x_1+x_2)+x_2(x_1+2x_2)\\ & =2x_1^2+x_1{x}_2+x_1{x}_2+2x_2^2\\ & =2x_1^2+2x_1{x}_2+2x_2^2\\ & =2(x_1^2+x_1{x}_2+x_2^2)\\ & =2\left[{\left(x_1+\frac{x_2}{2}\right)}^2+\frac{3x_2^2}{4}\right]>0\end{array}$$

Since ${\mathbf{x}}^{T}A\mathbf{x}>0$ for all nonzero $\mathbf{x}$, $A$ is positive definite.

Using eigenvalues

Let $A=\left[\begin{array}{cc}4& 2\\ 2& 3\end{array}\right]$

Find eigenvalues:

$$\begin{array}{rl}\det (A-\lambda I)& =\det \left[\begin{array}{cc}4-\lambda & 2\\ 2& 3-\lambda \end{array}\right]=0\\ (4-\lambda )(3-\lambda )-4& =0\\ 12-7\lambda +{\lambda }^2-4& =0\\ {\lambda }^2-7\lambda +8& =0\\ \lambda & =\frac{7\pm \sqrt{49-32}}{2}=\frac{7\pm \sqrt{17}}{2}\\ & \\ {\lambda }_1& \approx 5.56,{\lambda }_2\approx 1.44\end{array}$$

Since both eigenvalues are positive, $A$ is positive definite.

Using Sylvester’s Criterion

Let $A=\left[\begin{array}{ccc}3& 1& 0\\ 1& 2& 1\\ 0& 1& 3\end{array}\right]$

Check leading principal minors:

$$\begin{array}{rl}{M}_1& =\det [3]=3>0\\ M_2& =\det \left[\begin{array}{cc}3& 1\\ 1& 2\end{array}\right]=6-1=5>0\\ M_3& =\det \left[\begin{array}{ccc}3& 1& 0\\ 1& 2& 1\\ 0& 1& 3\end{array}\right]=3(6-1)-1(3-0)+0=15-3=12>0\end{array}$$

Since all leading principal minors are positive, $A$ is positive definite.

Indefinite matrix

Let $A=\left[\begin{array}{cc}1& 0\\ 0& -1\end{array}\right]$

The eigenvalues are ${\lambda }_1=1$ and ${\lambda }_2=-1$.

Since eigenvalues have different signs, $A$ is indefinite.

We can verify: for $\mathbf{x}=\left[\begin{array}{c}1\\ 0\end{array}\right]$: ${\mathbf{x}}^{T}A\mathbf{x}=1>0$

But for $\mathbf{x}=\left[\begin{array}{c}0\\ 1\end{array}\right]$: ${\mathbf{x}}^{T}A\mathbf{x}=-1<0$

Positive semidefinite (not definite)

Let $A=\left[\begin{array}{cc}1& 1\\ 1& 1\end{array}\right]$

Find eigenvalues:

$$\begin{array}{rl}\det (A-\lambda I)& =(1-\lambda {)}^2-1=0\\ {\lambda }^2-2\lambda & =0\\ \lambda (\lambda -2)& =0\\ & \\ {\lambda }_1& =2,{\lambda }_2=0\end{array}$$

Since one eigenvalue is zero and the other is positive, $A$ is positive semidefinite but not positive definite.