Neural Networks as Compositions of Functions

When you think of a neural network, imagine a machine that transforms data step by step, using a precise mathematical structure. At its core, a neural network is not just a collection of numbers or weights; it is a composition of functions. This means the network takes an input, applies a series of operations — each one transforming the data further — and produces an output. Each operation in this sequence is itself a function, and the overall effect is achieved by chaining these functions together.

This mathematical structure is what gives neural networks their power and flexibility. Each layer in a neural network performs two main actions. First, it applies a linear transformation to its input — this is typically a matrix multiplication with the layer’s weights, plus a bias. Immediately after, it applies a nonlinear activation function such as ReLU or sigmoid. This two-step process — linear map followed by nonlinearity — is repeated for every layer, making the network a deep composition of these alternating operations. The output of one layer becomes the input to the next, and the entire network can be viewed as a single function built by composing all these smaller functions in sequence.