Perceptron is the simplest neural network—only one neuron. For more complex tasks, we use a multilayer perceptron (MLP), which contains one or more hidden layers that allow the network to learn richer patterns.

An MLP consists of:

1. Input layer — receives data;
2. Hidden layers — extract patterns;
3. Output layer — produces predictions.

Each layer has multiple neurons; the output of one layer becomes the input of the next.

Layer Weights and Biases

Previously, a neuron stored its weights as a vector and bias as a scalar. A layer, however, contains many neurons, so its weights become a matrix, where each row stores the weights of one neuron. Biases for all neurons form a vector.

For a layer with 3 inputs and 2 neurons:

Here, $W_{ij}$ is the weight from the $j$-th input to the $i$-th neuron; $b_i$ is the bias of neuron $i$.

Forward Propagation

Forward propagation activates each neuron by computing a weighted sum, adding the bias, and applying the activation function.

Previously, a single neuron used:

[ z = W \cdot x + b ]

Now, since each row of (W) is one neuron's weight vector, performing matrix multiplication between the weight matrix and input vector automatically computes all neurons’ weighted sums at once.

To add the biases to the outputs of the respective neurons, a vector of biases should be added as well:

Finally, the activation function is applied to the result — sigmoid or ReLU, in our case. The resulting formula for forward propagation in the layer is as follows:

where $a$ is the vector of neuron activations (outputs).

Layer Class

Since MLPs are built from layers, we define a dedicated Layer class. Its attributes:

• inputs: input vector (n_inputs elements);
• outputs: raw neuron outputs (n_neurons elements);
• weights: weight matrix;
• biases: bias vector;
• activation_function: activation used in the layer.

Weights and biases are initialized with random values from a uniform distribution in ([-1, 1]). inputs and outputs are initialized as zero-filled NumPy arrays to ensure consistent shapes for later backpropagation.

class Layer:
    def __init__(self, n_inputs, n_neurons, activation_function):
        self.inputs = np.zeros((n_inputs, 1))
        self.outputs = np.zeros((n_neurons, 1))
        self.weights = ...
        self.biases = ...
        self.activation = activation_function

Forward Method

Forward propagation for a layer computes raw outputs and applies the activation:

def forward(self, inputs):
    self.inputs = np.array(inputs).reshape(-1, 1)
    # Raw outputs: weighted sum + bias
    self.outputs = ...
    # Apply activation
    return ...

Reshaping the input into a column vector ensures it multiplies correctly with the weight matrix and matches the expected dimensions throughout the network.