For this example, we build a neural network with one neuron for a binary classification task (e.g., spam detection). The neuron receives numerical features and outputs a value between 0 and 1, representing the probability that an email is spam (1) or ham (0).

Step-by-step:

1. Multiply each input by its weight;
2. Sum all weighted inputs;
3. Add a bias to shift the output;
4. Pass the result through a sigmoid activation, which converts it to the ((0,1)) range for probability output.

Neuron Class

A neuron needs to store its weights and bias making a class a natural way to group these related properties.

class Neuron:
    def __init__(self, n_inputs):
        self.weights = ...
        self.bias = ...

• weights: randomly initialized values (one per input);
• bias: a single random value. Both are drawn from a uniform distribution in ([-1, 1]) using np.random.uniform() to break symmetry.

Forward Propagation

The neuron’s activate() method computes the weighted sum and applies the sigmoid. The weighted sum uses the dot product of the weights and inputs:

input_sum_with_bias = np.dot(self.weights, inputs) + self.bias

Then we apply the activation to get the neuron’s final output.

Using np.dot() avoids loops and computes the entire weighted sum in one line. The sigmoid function then transforms this raw value into a probability:

def activate(self, inputs):
    input_sum_with_bias = ...
    output = ...
    return output

Sigmoid Activation

Given raw output (z), the sigmoid is:

It maps any number to ((0,1)), making it ideal for binary classification, where the neuron’s output must represent a probability.

Using this formula, sigmoid can be implemented as a simple function in Python:

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

The formula for the ReLU function is as follows, which basically sets the output equal to $z$ if it is positive and 0 otherwise:

def relu(z):
    return np.maximum(0, z)