Hessian Matrix

Definition

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a twice-differentiable scalar function.

The Hessian matrix of $f$ is the $n\times n$ matrix of second partial derivatives:

$$H_f(\mathbf{x})={\nabla }^{2}f(\mathbf{x})=\left[\begin{array}{cccc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_1\partial x_n}\\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}& \cdots & \frac{{\partial }^{2}f}{\partial x_2\partial x_n}\\ ⋮& ⋮& \ddots & ⋮\\ \frac{{\partial }^{2}f}{\partial x_n\partial x_1}& \frac{{\partial }^{2}f}{\partial x_n\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_n^2}\end{array}\right]$$

If $f$ is twice continuously differentiable, the Hessian is symmetric (by Schwarz’s theorem).

Example

Let $f(x_1,x_2)=x_1^2+3x_1{x}_2+2x_2^2$

First, compute the gradient:

$$\nabla f=\left[\begin{array}{c}2x_1+3x_2\\ 3x_1+4x_2\end{array}\right]$$

Then compute second derivatives:

$$\begin{array}{rl}{H}_f& =\left[\begin{array}{cc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}\\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}\end{array}\right]\\ & =\left[\begin{array}{cc}2& 3\\ 3& 4\end{array}\right]\end{array}$$

Note that the Hessian is constant (independent of $\mathbf{x}$) and symmetric.

The Hessian’s eigenvalues determine whether $f$ has a local minimum, maximum, or saddle point:

• If $H_f$ is positive definite: local minimum
• If $H_f$ is negative definite: local maximum
• If $H_f$ is indefinite: saddle point