Gradient Operator

Definition

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a scalar function of vector $\mathbf{x}={\left[\begin{array}{cccc}{x}_1& x_2& \cdots & x_n\end{array}\right]}^T$.

The gradient of $f$ with respect to $\mathbf{x}$ is the column vector of partial derivatives:

$$\nabla f(\mathbf{x})=\frac{\partial f}{\partial \mathbf{x}}=\left[\begin{array}{c}\frac{\partial f}{\partial x_1}\\ \frac{\partial f}{\partial x_2}\\ ⋮\\ \frac{\partial f}{\partial x_n}\end{array}\right]$$

The gradient points in the direction of steepest ascent of $f$.

Example

Let $f(x_1,x_2)=x_1^2+3x_1{x}_2+x_2^2$

$$\begin{array}{rl}\nabla f& =\left[\begin{array}{c}\frac{\partial f}{\partial x_1}\\ \frac{\partial f}{\partial x_2}\end{array}\right]\\ & =\left[\begin{array}{c}2x_1+3x_2\\ 3x_1+2x_2\end{array}\right]\end{array}$$

At $\mathbf{x}=\left[\begin{array}{c}1\\ 2\end{array}\right]$:

$$\nabla f\left(\left[\begin{array}{c}1\\ 2\end{array}\right]\right)=\left[\begin{array}{c}2(1)+3(2)\\ 3(1)+2(2)\end{array}\right]=\left[\begin{array}{c}8\\ 7\end{array}\right]$$