Learn Numerical Differentiation: Finite Differences

Numerical differentiation provides a way to estimate derivatives of functions when you do not have an explicit analytic form or when the function is only known at discrete points. The most common approach is to use finite difference formulas, which approximate the derivative by evaluating the function at nearby points.

Consider a function $f(x)$ that is smooth enough for differentiation. The derivative at a point $x$ is defined as the limit:

f'(x) = \lim_{h \to 0} \frac{f(x+h) - f(x)}{h}

Since you cannot take the limit numerically, you use a small but finite value of $h$ and construct formulas based on Taylor expansions.

Forward Difference Formula:

The forward difference uses the points $x$ and $x+h$ :

f'(x) \approx \frac{f(x+h) - f(x)}{h}

This formula is derived by expanding $f(x+h)$ in a Taylor series and neglecting higher-order terms.

Backward Difference Formula:

The backward difference uses the points $x$ and $x-h$ :

f'(x) \approx \frac{f(x) - f(x-h)}{h}

This is similarly derived by expanding $f(x-h)$ .

Central Difference Formula:

The central difference uses both $x+h$ and $x-h$ :

f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}

The central difference is generally more accurate, as it cancels out some of the leading error terms from the Taylor expansion.


              1234567891011121314151617181920212223242526272829
            
# Python implementation of finite difference methods

def forward_difference(f, x, h):
    """Estimate derivative using the forward difference formula."""
    return (f(x + h) - f(x)) / h

def backward_difference(f, x, h):
    """Estimate derivative using the backward difference formula."""
    return (f(x) - f(x - h)) / h

def central_difference(f, x, h):
    """Estimate derivative using the central difference formula."""
    return (f(x + h) - f(x - h)) / (2 * h)

# Example: Estimate derivative of f(x) = x**2 at x = 1
def f(x):
    return x ** 2

x0 = 1.0
h = 1e-5

fwd = forward_difference(f, x0, h)
bwd = backward_difference(f, x0, h)
ctr = central_difference(f, x0, h)

print("Forward difference:", fwd)
print("Backward difference:", bwd)
print("Central difference:", ctr)
# The exact derivative at x=1 is 2

Each finite difference formula has its own sources of error and level of accuracy.

The forward and backward difference formulas are both first-order accurate in $h$ . This means the error scales linearly with the step size: halving $h$ roughly halves the error. The leading error term comes from the second derivative of $f(x)$ , multiplied by $h/2$ ;
The central difference formula is second-order accurate in $h$ . Its error scales with $h^2$ : halving $h$ reduces the error by a factor of four. This higher accuracy results from the cancellation of some error terms in the Taylor expansion;
All methods are affected by two main sources of error: truncation error (from neglecting higher-order terms in the Taylor expansion) and round-off error (from the finite precision of floating-point arithmetic). If $h$ is too large, truncation error dominates; if $h$ is too small, round-off error increases due to subtractive cancellation. Choosing an optimal $h$ requires balancing these effects;
Central differences are generally preferred for interior points due to their higher accuracy, but forward or backward differences are necessary at boundaries where points on only one side are available.

Everything was clear?

Thanks for your feedback!

Section 2. Chapter 1

Ask AI

Ask anything or try one of the suggested questions to begin our chat

Swipe to show menu