Partial derivatives

We already know that the derivative of a function represents the rate of change of its output corresponding to an infinitesimally small change of its input. However, when we work with a multivariable function, its input might change in more than one direction — since it is specified by more than one coordinate — as may its output. In this topic, we will take a look at partial derivatives and how can they help us study change in multivariable functions, building to the concept of gradient.

Different perspectives

As we learned previously, the derivative of a function $f(x)$ at a given point represents the slope of the line tangent to $f(x)$ at that point, and it tells us how much $f(x)$ is increasing or decreasing within a very small interval $(x - \Delta x, \; x + \Delta x)$ as $\Delta x \to0$.

Now, let's consider an $f:\mathbb R^2 \to \mathbb R$; for example:

$$f(x,y)=\frac{7xy}{e^{x^2 + y^2}}.$$

Since $f$ now takes two independent variables as input, we must take into account how changes in each one contribute to the change in$f$.

Let's plot $f(x,y)$ in $\mathbb R^3$ using Cartesian coordinates using $z=f(x,y)$.

We can see that the rate of change of $z$ as $x$ increases depends not only on the value of $x$, but also on $y$.

For example, let's say $y=1$ while $x$ increases:

And now, let's say $y=-1$ while $x$ increases:

In the same manner, when increasing $y$ from any $y_0$, $z$ might either increase or decrease depending on the value of $x$.

For instance, let's hold $x=-1$ while $y$ increases:

This process of changing one variable while holding all the others constant results in what we call a partial derivative.

The rate of change of $f(x,y)$ with respect to $x$ can be denoted as

$$\frac{\partial f}{\partial x}(x,y),$$while the rate of change of $f(x,y)$ with respect to $y$ can be denoted as

$$\frac{\partial f}{\partial y}(x,y).$$As with traditional one-dimensional derivatives, we have more than one notation:

$$\frac{\partial f}{\partial x}(x,y) \equiv f_x (x,y) \quad ; \quad \frac{\partial f}{\partial y}(x,y) \equiv f_y (x,y).$$

For example, let's define $g: \mathbb{R}^2 \to \mathbb{R}$ by

$$g(x,y) = x^3 + xy-y.$$

In order to determine the partial derivative of $g$ with respect to $x$, we need to treat $y$ as a constant. Then, we can differentiate our expression normally, as if it only depended on $x$:

$$(x^3 + kx-k)' = (x^3)' + (kx)' - (k)' = 3x^2 + k - 0$$Then, we have

$$\frac{\partial g}{\partial x}(x,y) = g_x(x,y) = 3x^2 +y$$Conversely, to determine the partial derivative of $g$ with respect to $y$, we must treat $x$ as a constant.

Then,

$$\frac{\partial g}{\partial y}(x,y) = g_y(x,y) = x - 1$$For the previous example, we have the partial derivatives

$$f(x,y)=\frac{7xy}{e^{x^2 + y^2}} \implies \begin{cases} \frac{\partial f}{\partial x} = - \frac{7y(2x^2-1)}{e^{x^2 + y^2}}\\ \\ \frac{\partial f}{\partial y} = - \frac{7x(2y^2-1)}{e^{x^2 + y^2}} \end{cases}$$

More formally, given a function $f: \mathbb R^n \to \mathbb R$ we can define its partial derivative with respect to $x_i$ at the point $\bold{x} \in \mathbb R^n$ as:

$$\frac{\partial f}{\partial x_i}(\bold{x})=\lim \limits_{\Delta x_i \to 0} \frac{f(\bold{x}+\Delta x_i \cdot \bold{e_i}) - f(\bold{x})}{\Delta x_i} \qquad ; \quad \bold{x}=(x_1,\dots,x_i,\dots,x_n).$$Since $\mathbf{e_i}$ is the $i^{\text{th}}$ basis vector, we have$$\frac{\partial f}{\partial x_i}(\bold{x})=\lim \limits_{\Delta x_i \to 0} \frac{f(x_1,\dots,x_i +\Delta x_i ,\dots,x_n) - f(x_1,\dots,x_i,\dots,x_n)}{\Delta x_i}.$$

Keep in mind that the value of a partial derivative depends on the coordinate system that we choose to express our function in. For example, if we decided to use polar coordinates to describe the set $\mathbb{R}^2$, the partial derivatives with respect to $r$ and $\theta$ wouldn't be the same as the partial derivatives we get from the Cartesian coordinates $x$ and $y$. This makes sense because a change in radius length or angle isn't the same as a change along the $x$ or $y$ axis.

Gradient

Let's say

$$f(x,y,z)=x^2y+yz+xyz^2$$

Then, we have:

$$\begin{matrix*}[l] \frac{\partial f}{\partial x}(x,y,z) = 2xy + yz^2 \\ \\ \frac{\partial f}{\partial y}(x,y,z) = x^2 + z + xz^2 \\ \\ \frac{\partial f}{\partial z}(x,y,z) = y + 2xyz \end{matrix*}$$We can define a vector-valued function by placing each partial derivative inside a vector in its corresponding coordinate:

$$\nabla f(x,y,z)=\begin{pmatrix} \frac{\partial f}{\partial x}(x,y,z)\\ \\ \frac{\partial f}{\partial y}(x,y,z)\\ \\ \frac{\partial f}{\partial z}(x,y,z) \end{pmatrix}$$This vector field is called a gradient. It is represented with the del operator $\nabla$, also known as nabla.

Sometimes, the del operator is defined separately as:

$$\nabla= \left ( \frac{\partial}{\partial x_0},\dots,\frac{\partial}{\partial x_i},\dots,\frac{\partial}{\partial x_n} \right)$$So, in Cartesian coordinates in $\mathbb R^3$ it would be:

$$\nabla= \left ( \frac{\partial}{\partial x},\frac{\partial}{\partial y},\frac{\partial}{\partial z} \right)$$And, since $f$ is a scalar-valued function, by multiplying it times nabla,

$$\nabla f(x,y,z) = \left ( \frac{\partial}{\partial x},\frac{\partial}{\partial y},\frac{\partial}{\partial z} \right) \cdot f(x,y,z) = \left ( \frac{\partial f}{\partial x},\frac{\partial f}{\partial y},\frac{\partial f}{\partial z} \right)$$We obtain its corresponding gradient.

Now, how can we express the partial derivatives if there is more than one output?

Higher order derivatives

Just like with ordinary derivatives, we can also have higher order partial derivatives. To obtain the second-order partial derivative of $f$ with respect to $x$, we just take the partial derivative with respect to $x$ of the first partial derivative with respect to $x$ of $f$:

$${\frac {\partial ^{2}f}{\partial x^{2}}} = \frac{\partial}{\partial x} \left ( \frac{\partial f}{\partial x} \right ) \equiv f_{xx}$$However, if instead we take the partial derivative with respect to $y$ of the partial derivative with respect to $x$ of $f$, we get the so called mixed partial derivative:

$${\frac {\partial ^{2}f}{\partial y \partial x}} = \frac{\partial}{\partial y} \left ( \frac{\partial f}{\partial x} \right ) \equiv f_{xy}$$

One neat fact about mixed partial derivatives is that, if $\dfrac{\partial f}{\partial x}$, $\dfrac{\partial f}{\partial y}$, ${\dfrac {\partial ^{2}f}{\partial y \partial x}}$ and ${\dfrac {\partial ^{2}f}{\partial x \partial y}}$ are continuous, then the following is true:

$${\frac {\partial ^{2}f}{\partial y \partial x}} ={\frac {\partial ^{2}f}{\partial x \partial y}}$$

For example, let's calculate the second order derivatives for the function $f(x,y) = x^2y^2 -\sin(x) + \cos(y)$. We can start with the calculation of first order derivatives.

$$\frac{\partial f}{\partial x} = 2xy^2 - \cos(x),$$

$$\frac{\partial f}{\partial y} = 2x^2y - \sin(y),$$$$\frac{\partial^2 f}{\partial y \partial x} = \frac{\partial}{\partial y }(2xy^2 - \cos(x)) = 4xy,$$$$\frac{\partial^2 f}{\partial x \partial y} = \frac{\partial}{\partial x}(2x^2y - \sin(y)) = 4xy,$$$$\frac{\partial^2 f}{\partial x^2} = \frac{\partial}{\partial x}(2xy^2 - \cos(x)) = 2y^2 + \sin(x),$$$$\frac{\partial^2 f}{\partial y^2} = \frac{\partial}{\partial y}(2x^2y - \sin(y)) = 2x^2 - \cos(y).$$

Conclusion

To sum up, in this topic we have learned:

• The partial derivative of a function with respect to any of its input variables is obtained by differentiating in one variable while all the others remain constant.
• The vector-valued gradient function $\nabla f$ of a scalar-valued function $f$ can be obtained by grouping all the partial derivatives of $f$ in a single vector.
• We can obtain higher order partial derivatives by differentiating a partial derivative with respect to one of its arguments.