Learn Introductions to Derivatives | Mathematical Analysis

Definition

A derivative is a measure of how a function changes as its input changes. It represents the rate of change of the function and is fundamental in analyzing trends, optimizing processes, and predicting behavior in fields such as physics, economics, and machine learning.

The Limit Definition of a Derivative

The derivative of a function $f(x)$ at a specific point $x = a$ is given by:

\lim_{h \rarr 0} \frac{f(x + h) - f(x)}{h}

This formula tells us how much $f(x)$ changes when we make a tiny step $h$ along the x-axis. The smaller $h$ becomes, the closer we get to the instantaneous rate of change.

Basic Derivative Rules

Power Rule

If a function is a power of $x$ , the derivative follows:

\frac{d}{dx}x^n=nx^{n-1}

This means that when differentiating, we bring the exponent down and reduce it by one:

\frac{d}{dx}x^3=3x^2

Constant Rule

The derivative of any constant is zero:

\frac{d}{dx}C=0

For example, if $f(x) = 5$ , then:

\frac{d}{dx}5=0

Sum & Difference Rule

The derivative of a sum or difference of functions follows:

\frac{d}{dx} \left[ f(x) \pm g(x) \right] = f'(x) \pm g'(x)

For example, differentiating separately:

\frac{d}{dx}(x^3 + 2x) = 3x^2 + 2

Product & Quotient Rules

Product Rule

If two functions are multiplied, the derivative is found as follows:

\frac{d}{dx}[f(x)g(x)] = f'(x)g(x) + f(x)g'(x)

This means we differentiate each function separately and then sum their products. If $f(x)=x^2$ and $g(x) = e^x$ , then:

\frac{d}{dx}[x^2e^x] = 2xe^x + x^3e^x

Quotient Rule

When dividing functions, use:

\frac{d}{dx} \left[ \frac{f(x)}{g(x)} \right] = \frac{f'(x)g(x) - f(x)g'(x)}{g(x)^2}

If $f(x)=x^2$ and $g(x)=x+1$ , then:

\frac{d}{dx} \left[ \frac{x^2}{x + 1} \right] = \frac{2x(x+1) - x^2(1)}{(x+1)^2}

Chain Rule: Differentiating Composite Functions

When differentiating nested functions, use:

\frac{d}{dx} f(g(x)) = f'(g(x)) \cdot g'(x)

For example, if $y = (3x + 2)^5$ , then:

\frac{d}{dx}(3x+2)^5 = 5(3x+2)^4 \cdot 3 = 15(3x+2)^4

This rule is essential in neural networks and machine learning algorithms.

Exponential Chain Rule Example:

When you're differentiating something like:

y =e^{2x^2}

You're dealing with a composite function:

Outer function: $e^u$
Inner function: $u = 2x^2$

Apply the chain rule step-by-step:

\frac{d}{dx}2x^2=4x

Then multiply by the original exponential:

\frac{d}{dx}\left( e^{2x^2} \right) = 4x \cdot e^{2x^2}

Study More

In machine learning and neural nets, this shows up when working with exponential activations or loss functions.

Logarithmic Chain Rule Example:

Let's differentiate $\ln(2x)$ . Again, it's a composite function — log on the outside, linear on the inside.

Differentiate the inner part:

\frac{d}{dx}(2x)=2

Now apply the chain rule to the log:

\frac{d}{dx}\ln(2x) = \frac{1}{2x} \cdot 2

Which simplifies to:

\frac{d}{dx}\ln(2x) = \frac{2}{2x} = \frac{1}{x}

Note

Even if you’re differentiating $\ln(kx)$ , the result is always $\frac{\raisebox{1pt}{$1$}}{\raisebox{-1pt}{$x$}}$ because the constants cancel out.

Special Case: Derivative of the Sigmoid Function

The sigmoid function is commonly used in machine learning:

\sigma(x) = \frac{1}{1+x^{-x}}

Its derivative plays a key role in optimization:

\sigma'(x) = \sigma(x)(1 - \sigma(x))

If $f(x) = \frac{\raisebox{1pt}{$1$}}{\raisebox{-3pt}{$1 + e^{-x}$}}$ , then:

f'(x) = \frac{e^{-x}}{(1 + e^{-x})^2}

This formula ensures that gradients remain smooth during training.

Everything was clear?

Thanks for your feedback!

Section 3. Chapter 3

Ask AI

Ask anything or try one of the suggested questions to begin our chat

Suggested prompts:

Can you explain the limit definition of a derivative with a simple example?

How do the product, quotient, and chain rules differ in practice?

Can you show how to find the derivative of a more complex function using these rules?

Awesome!

Completion rate improved to 1.96

Swipe to show menu