3 Derivatives and Differentials of Functions

3.1 Definition and Calculation of Derivatives

Differential

A point $x_0$ is called an interior point of $I$, if there exists $\delta > 0$ such that

$$\begin{equation*} \forall x,\ |x - x_0| < \delta \Rightarrow x \in I. \end{equation*}$$

Let $x_0$ be an interior point of $I$. A function $f: I \to \mathbb{R}$ is said to be differentiable at $x_0$ if there exists a linear function $L: \mathbb{R} \to \mathbb{R}$ such that

$$\begin{equation*} f(x_0 + h) = f(x_0) + L(h) + o(h), \quad h \to 0. \end{equation*}$$

In this case, $L$ is called the differential of $f$ at $x_0$, denoted by $df(x_0)$.

Derivative

For a one-variable function, the differential $df(x_0)$ is a linear function, that is, a proportional function whose proportionality constant is denoted by $f'(x_0)$. This $f'(x_0)$ is called the derivative of $f$ at $x_0$.

$$\begin{equation*} df(x_0)(h) = f'(x_0)h. \end{equation*}$$ $$\begin{equation*} \lim_{h \to 0} \frac{f(x_0 + h) - f(x_0) - f'(x_0)h}{h} = \lim_{h \to 0} \frac{o(h)}{h} = 0. \end{equation*}$$

The derivative is defined as:

$$\begin{equation*} f'(x_0) = \lim_{h \to 0} \frac{f(x_0 + h) - f(x_0)}{h}. \end{equation*}$$

If this limit exists, we say that $f$ is differentiable at $x_0$.

• Traditional notation: Let $\Delta x = h$, $\Delta y = f(x_0 + \Delta x) - f(x_0)$, then

$$\begin{equation*} f'(x_0) = \lim_{h \to 0} \frac{\Delta y}{\Delta x} = \left. \frac{dy}{dx} \right|_{x_0}. \end{equation*}$$

• Lagrange notation: $f'(x_0).$

• Leibniz notation: $\displaystyle \frac{dy}{dx}.$

Theorem 3.1. Let $f, g$ be differentiable at $x_0$. Then $f \pm g$ and $fg$ are differentiable at $x_0$, and

$$\begin{equation*} (f \pm g)'(x_0) = f'(x_0) \pm g'(x_0), \end{equation*}$$ $$\begin{equation*} (fg)'(x_0) = g(x_0) f'(x_0) + f(x_0) g'(x_0). \end{equation*}$$

If $g(x_0) \neq 0$, then

$$\begin{equation*} \left( \frac{f}{g} \right)'(x_0) = \frac{f'(x_0) g(x_0) - f(x_0) g'(x_0)}{[g(x_0)]^2}. \end{equation*}$$

Theorem 3.2. Suppose $f$ is differentiable at $x_0$, and $g$ is differentiable at $y_0 = f(x_0)$. Then the composite function $g \circ f$ is differentiable at $x_0$, and

$$\begin{equation*} \mathrm{d}(g \circ f)(x_0) = \mathrm{d}g(y_0) \circ \mathrm{d}f(x_0). \end{equation*}$$

Equivalently,

$$\begin{equation*} (g \circ f)'(x_0) = g'(y_0) \cdot f'(x_0) = g'(f(x_0)) f'(x_0). \end{equation*}$$

Theorem 3.3. Suppose $f$ has a continuous inverse function. If $f$ is differentiable at $x_0$ and $f'(x_0) \neq 0$, then $f^{-1}$ is differentiable at $y_0 = f(x_0)$, and

$$\begin{equation*} \mathrm{d}(f^{-1})(y_0) = (\mathrm{d}f(x_0))^{-1}. \end{equation*}$$

Equivalently,

$$\begin{equation*} (f^{-1})'(y_0) = \frac{1}{f'(x_0)}. \end{equation*}$$

Examples

1. $d(x^n)=n x^{n-1}dx\qquad(n\in\mathbb{R})$.

2. $d(e^x)=e^x\,dx,\qquad d(\ln x)=\frac{1}{x}dx\ (x>0)$.

3. $d(a^x)=a^x\ln a\,dx,\qquad d(\log_a x)=\frac{1}{x\ln a}dx\ (x>0)$.

4. $d(e^{u})=e^{u}u'dx,\qquad d(\ln u)=\frac{u'}{u}dx$.

5. $d(\sin x)=\cos x\,dx,\quad d(\cos x)=-\sin x\,dx,\quad d(\tan x)=\sec^2 x\,dx$.

6. $d(\cot x)=-\csc^2 x\,dx,\quad d(\sec x)=\sec x\tan x\,dx,\quad d(\csc x)=-\csc x\cot x\,dx$.

7. $d(\arcsin x)=\frac{1}{\sqrt{1-x^2}}dx,\quad d(\arccos x)=-\frac{1}{\sqrt{1-x^2}}dx$.

8. $d(\arctan x)=\frac{1}{1+x^2}dx$.

3.2 Higher-Order Derivatives

Higher-Order Derivative

If a function $f$ is differentiable everywhere in an interval $I$, then the function

$$\begin{equation*} f^{(1)} := f' : I \to \mathbb{R} \end{equation*}$$

is called the first derivative of $f$. We denote $f^{(0)} := f$.

By induction, if the $n$-th derivative $f^{(n)}$ of $f$ exists, and $f^{(n)}$ is differentiable at $x_0$, then we define

$$\begin{equation*} f^{(n+1)}(x_0) = \big(f^{(n)}\big)'(x_0). \end{equation*}$$

We call $f^{(n+1)}(x_0)$ the $(n+1)$-th derivative of $f$ at $x_0$. The second and third derivatives are denoted by $f''$ and $f'''$, respectively.

• We also use the following notations:

$$\begin{equation*} f \in \mathcal{C}^n(I) \quad \text{means } f^{(n)} \text{ is continuous on } I; \end{equation*}$$ $$\begin{equation*} f \in \mathcal{C}^\infty(I) \quad \text{means } f \text{ has derivatives of all orders on } I. \end{equation*}$$

Theorem 3.4. Let $f, g$ have $n$-th derivatives at $x_0$. Then $\lambda f + \mu g$ and $fg$ also have $n$-th derivatives at $x_0$, and we have:

$$\begin{equation*} (\lambda f + \mu g)^{(n)}(x_0) = \lambda f^{(n)}(x_0) + \mu g^{(n)}(x_0) \end{equation*}$$ $$\begin{equation*} (fg)^{(n)}(x_0) = \sum_{k=0}^{n} C_n^k\, f^{(k)}(x_0)\, g^{(n-k)}(x_0) \end{equation*}$$

Theorem 3.5. If $f$ is $n$-times differentiable at $x_0$, and $g$ is $n$-times differentiable at $y_0 = f(x_0)$, then the composite function $g \circ f$ is $n$-times differentiable at $x_0$.

Theorem 3.6. Let $f, g$ be $n$-times differentiable at $x_0$, and suppose $g(x_0) \neq 0$. Then $\dfrac{f}{g}$ is $n$-times differentiable at $x_0$.

Theorem 3.7. The curvature is the reciprocal of the radius of curvature:

$$\begin{equation*} \kappa = \frac{\Big| \det \begin{pmatrix} x'(t) & x''(t) \\ y'(t) & y''(t) \end{pmatrix} \Big|} {\big[(x'(t))^2 + (y'(t))^2\big]^{3/2}} \end{equation*}$$

3.3 Differential Mean Value Theorems

Local Extremum

A point $x_0 \in I$ is called a local maximum of $f$ if there exists a neighborhood $V$ of $x_0$ such that

$$\begin{equation*} \forall x \in V \cap I,\quad f(x) \le f(x_0). \end{equation*}$$

The definition of a local minimum is similar.

Critical Point

A point $x_0$ is called a critical point of $f$ if

$$\begin{equation*} f'(x_0) = 0. \end{equation*}$$

Theorem 3.8. If $f$ is differentiable at a local extremum $x_0$, then $x_0$ is a critical point of $f$.

Theorem 3.9 (Rolle's Theorem). Suppose $f$ is continuous on $[a,b]$, differentiable on $(a,b)$, and $f(a) = f(b)$. Then there exists some $\xi \in (a,b)$ such that

$$\begin{equation*} f'(\xi) = 0. \end{equation*}$$

Corollary 1. Suppose $f$ is differentiable on the open interval $(a,b)$, and

$$\begin{equation*} \lim_{x \to a^+} f(x) = \lim_{x \to b^-} f(x) = A \in \mathbb{R} \cup \{\pm\infty\}. \end{equation*}$$

Then there exists some $\xi \in (a,b)$ such that

$$\begin{equation*} f'(\xi) = 0. \end{equation*}$$

Theorem 3.10 (Cauchy's Mean Value Theorem). Let $-\infty \le t_1 < t_2 \le +\infty$, and let $\alpha, \beta, A, B \in \mathbb{R}$. Suppose $x(t)$ and $y(t)$ are differentiable on the open interval $(t_1, t_2)$, and

$$\begin{equation*} \lim_{t \to t_1^+} x(t) = x_1,\quad \lim_{t \to t_2^-} x(t) = x_2,\quad \lim_{t \to t_1^+} y(t) = y_1,\quad \lim_{t \to t_2^-} y(t) = y_2. \end{equation*}$$

Then there exists some $\xi \in (t_1, t_2)$ such that

$$\begin{equation*} x'(\xi)(y_2 - y_1) = y'(\xi)(x_2 - x_1). \end{equation*}$$

Theorem 3.11 (Lagrange's Mean Value Theorem). Suppose $f$ is continuous on the closed interval $[a,b]$ and differentiable on the open interval $(a,b)$. Then there exists some $\xi \in (a,b)$ such that

$$\begin{equation*} f'(\xi) = \frac{f(b) - f(a)}{b - a}. \end{equation*}$$

Theorem 3.12 (Darboux's Theorem). Suppose $f$ is differentiable on an interval $I$. Then $f'(I)$ is an interval. In particular, if $x_1, x_2 \in I$ satisfy

$$\begin{equation*} f'(x_1) < 0 < f'(x_2), \end{equation*}$$

then there exists some $\xi$ between $x_1$ and $x_2$ such that

$$\begin{equation*} f'(\xi) = 0. \end{equation*}$$

Corollary 1. If $f$ is differentiable on an interval $I$, and

$$\begin{equation*} f'(x) \ne 0 \quad (\forall x \in I), \end{equation*}$$

then $f$ is strictly monotonic on $I$.

3.4 Using Derivatives to Analyze Functions

Monotonicity of Function

Theorem 3.13. Suppose $f$ is continuous on an interval $I$ and differentiable in the interior of $I$. Then:

$$\begin{equation*} \forall x \in I,\quad f'(x) \ge 0\ \ (\text{resp. } f'(x) \le 0) \end{equation*}$$

​ iff $f$ is non-decreasing (resp. non-increasing) on $I$.

$$\begin{equation*} \forall x \in I,\quad f'(x) = 0 \end{equation*}$$

​ iff $f$ is constant on $I$.

$$\begin{equation*} \forall x \in I,\quad f'(x) > 0\ \ (\text{resp. } f'(x) < 0) \end{equation*}$$

​ implies that $f$ is strictly increasing (resp. strictly decreasing) on $I$.

Strict Local Extremum

Theorem 3.14. Assume $f$ is differentiable at $x_0$ and

$$\begin{equation*} f'(x_0) = 0. \end{equation*}$$

• If $f''(x_0) > 0$, then $x_0$ is a strict local minimum of $f$.
• If $f''(x_0) < 0$, then $x_0$ is a strict local maximum of $f$.

Theorem 3.15. Assume $f$ is $2n$-times differentiable at $x_0$, where $n$ is a positive integer, and

$$\begin{equation*} f'(x_0) = \cdots = f^{(2n-1)}(x_0) = 0. \end{equation*}$$

• If $f^{(2n)}(x_0) > 0$, then $x_0$ is a strict local minimum of $f$.
• If $f^{(2n)}(x_0) < 0$, then $x_0$ is a strict local maximum of $f$.

Theorem 3.16. Assume $f$ is $(2n+1)$-times differentiable at $x_0$, where $n$ is a positive integer, and

$$\begin{equation*} f'(x_0) = \cdots = f^{(2n)}(x_0) = 0. \end{equation*}$$

• If $f^{(2n+1)}(x_0) > 0$, then $f$ is strictly increasing in a neighborhood of $x_0$.
• If $f^{(2n+1)}(x_0) < 0$, then $f$ is strictly decreasing in a neighborhood of $x_0$.

3.5 Convexity of Functions

Convex Function

A function $f$ is called convex on an interval $I$ if for all $x_1, x_2 \in I$ and all $t$ with $0 < t < 1$, we have

$$\begin{equation*} f((1-t)x_1 + t x_2) \le (1-t) f(x_1) + t f(x_2). \end{equation*}$$

If equality occurs only when $x_1 = x_2$, then $f$ is called strictly convex.

Theorem 3.17. Let $f$ be a convex function on the interval $I$. Then for any $x_1, x_2, \cdots, x_n \in I$, and $\lambda_1, \lambda_2, \cdots, \lambda_n > 0$, with $\lambda_1 + \lambda_2 + \cdots + \lambda_n = 1$, we have

$$\begin{equation*} f\left(\sum_{i=1}^{n} \lambda_i x_i\right) \leqslant \sum_{i=1}^{n} \lambda_i f(x_i). \end{equation*}$$

If $f$ is a strictly convex function on $I$, then when $x_1, x_2, \cdots, x_n$ are not all equal, we have

$$\begin{equation*} f\left(\sum_{i=1}^{n} \lambda_i x_i\right) < \sum_{i=1}^{n} \lambda_i f(x_i). \end{equation*}$$

Theorem 3.18. The function $f$ is a convex function on the interval $I$ if and only if for any $(x_1, x_2) \subset I$ and any $x \in (x_1, x_2)$, we have

$$\begin{equation*} \frac{f(x) - f(x_1)}{x - x_1} \leqslant \frac{f(x_2) - f(x_1)}{x_2 - x_1} \leqslant \frac{f(x_2) - f(x)}{x_2 - x}. \end{equation*}$$

Corollary 1. Let $f: I \to \mathbb{R}$ be a convex function. If $x_0$ is an interior point of $I$ (i.e., $x_0 \in \text{int}(I)$), then $f$ is continuous at $x_0$.

Corollary 2. Let $f$ be convex on an open interval $(a, b)$. For every $x \in (a, b)$, both the left-hand derivative $f'_-(x)$ and the right-hand derivative $f'_+(x)$ exist and are finite.

Theorem 3.19. Let $f$ be continuous on $[a, b]$ and differentiable on $(a, b)$, then a necessary and sufficient condition for $f$ to be a convex function (strictly convex function) on $[a, b]$ is that $f'$ is increasing (strictly increasing) on $(a, b)$.

3.6 L'Hospital's Rule

Theorem 3.20 ($\frac{0}{0}$ Form). Let $f, g$ be differentiable on the interval $(a,b)$. Assume that

$$\begin{equation*} f(x) = o(1),\quad g(x)=o(1),\quad x\to a,\quad g'(x)\ne 0. \end{equation*}$$

If

$$\begin{equation*} \lim_{x\to a} \frac{f'(x)}{g'(x)} = A \in \mathbb{R}\cup\{\pm\infty\}, \end{equation*}$$

then

$$\begin{equation*} \lim_{x\to a} \frac{f(x)}{g(x)} = A. \end{equation*}$$

Theorem 3.21 ($\frac{\infty}{\infty}$ Form). Assume that $g(x)\to\infty$ as $x\to a$, and $g'(x)\neq 0$. If

$$\begin{equation*} \lim_{x\to a} \frac{f'(x)}{g'(x)} = A \in \mathbb{R}\cup\{\pm\infty\}, \end{equation*}$$

then

$$\begin{equation*} \lim_{x\to a} \frac{f(x)}{g(x)} = A. \end{equation*}$$

Confirm that $g'(x)\neq 0$ on a neighborhood of the point

3.7 The Taylor Formula and Its Analytical Applications

Definition of the $n$-th Order Taylor Polynomial

Assume that $f$ has $n$ derivatives at $x_0$. The $n$-th order Taylor polynomial of $f$ at $x_0$ is defined by

$$\begin{equation*} T_{f,x_0,n}(h) = f(x_0) + f'(x_0) h + \frac{f''(x_0)}{2} h^2 + \cdots + \frac{f^{(n)}(x_0)}{n!} h^n . \end{equation*}$$

We call $T_{f,x_0,n}(h)$ the $n$-th order Taylor polynomial of $f$ at $x_0$.

Taylor’s Theorem with Peano Remainder

If $f$ has $n$ derivatives at $x_0$, then a polynomial

$$\begin{equation*} P_n(h)=a_0+a_1 h+\cdots+a_n h^n \end{equation*}$$

satisfies

$$\begin{equation*} f(x_0+h)=P_n(h)+o(h^n),\qquad h\to 0, \end{equation*}$$

if and only if $P_n(h)$ is the $n$-th order Taylor polynomial of $f$ at $x_0$.

Taylor's Theorem with Lagrange Remainder

Assume that $f$ is continuous on an interval $I$ and $(n+1)$ times differentiable on the interior of $I$. For any interior point $x_0\in I$ and any $x\in I$, there exists a point $\xi$ strictly between $x$ and $x_0$ such that

$$\begin{equation*} f(x) = T_{f,x_0,n}(x - x_0) + \frac{f^{(n+1)}(\xi)}{(n+1)!}\,(x-x_0)^{\,n+1}. \end{equation*}$$

Examples

$$\begin{equation*} e^{x} = 1 + x + \frac{x^{2}}{2} + \cdots + \frac{x^{n}}{n!} + o(x^{n}),\qquad x\to 0. \end{equation*}$$

2. $$\begin{equation*} \sin x = x - \frac{x^{3}}{3!} + \cdots + (-1)^{n}\frac{x^{2n+1}}{(2n+1)!} + o(x^{2n+2}),\qquad x\to 0, \end{equation*}$$
3. $$\begin{equation*} \cos x = 1 - \frac{x^{2}}{2!} + \cdots + (-1)^{n}\frac{x^{2n}}{(2n)!} + o(x^{2n+1}),\qquad x\to 0, \end{equation*}$$
4. $$\begin{equation*} \ln(1+x) = x- \frac{x^{2}}{2} + \cdots + (-1)^{n-1}\frac{x^{n}}{n} + o(x^{n}),\qquad x\to 0. \end{equation*}$$
5. $$\begin{equation*} (1+x)^{\alpha} = 1 + \alpha x + \frac{\alpha(\alpha-1)}{2!}x^{2} + \cdots + \frac{\alpha(\alpha-1)\cdots(\alpha-n+1)}{n!}x^{n} + o(x^{n}),\qquad x\to 0. \end{equation*}$$
6. $$\begin{equation*} \arctan x = x - \frac{x^{3}}{3} + \cdots + (-1)^{n}\frac{x^{2n+1}}{2n+1} + o(x^{2n + 2}),\qquad x\to 0, \end{equation*}$$