Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.Leer Challenge: Training the Perceptron | Neural Network from Scratch
Introduction to Neural Networks
course content

Cursusinhoud

Introduction to Neural Networks

Introduction to Neural Networks

1. Concept of Neural Network
What is a Neural Network?Applications of Deep Learning in the Real WorldNeural Networks or Traditional ModelsNeural Network StructureWhat is a Neuron?Activation FunctionsForward and Backward Propagation
2. Neural Network from Scratch
Single Neuron ImplementationChallenge: Creating a NeuronPerceptron LayersChallenge: Creating a PerceptronForward PropagationLoss FunctionBackward PropagationBackpropagation ImplementationModel TrainingChallenge: Training the PerceptronModel EvaluationChallenge: Evaluating the PerceptronNeural Network with scikit-learn
3. Conclusion
Other types of Neural NetworksHyperparameter TuningChallenge: Automatic Hyperparameter TuningQuiz: Neural Networks EssentialsSummary

book
Challenge: Training the Perceptron

Before proceeding with training the perceptron, keep in mind that it uses the binary cross-entropy loss function discussed earlier. The final key concept before implementing backpropagation is the formula for the derivative of this loss function with respect to the output activations, an. Below are the formulas for the loss function and its derivative:

To verify that the perceptron is training correctly, the fit() method also prints the average loss at each epoch. This is calculated by averaging the loss over all training examples in that epoch:

python

Finally, the formulas for computing gradients are as follows:

The sample training data (X_train) along with the corresponding labels (y_train) are stored as NumPy arrays in the utils.py file. Additionally, instances of the activation functions are also defined there:

python
Taak

Swipe to start coding

  1. Compute the following gradients: dz, d_weights, d_biases, and da_prev in the backward() method of the Layer class.
  2. Compute the output of the model in the fit() method of the Perceptron class.
  3. Compute da (dan) before the loop, which is the gradient of the loss with respect to output activations.
  4. Compute da and perform backpropagation in the loop by calling the appropriate method for each of the layers.

If you implemented training correctly, given the learning rate of 0.01, the loss should steadily decrease with each epoch.

Oplossing

Switch to desktopSchakel over naar desktop voor praktijkervaringGa verder vanaf waar je bent met een van de onderstaande opties
Was alles duidelijk?

Hoe kunnen we het verbeteren?

Bedankt voor je feedback!

Sectie 2. Hoofdstuk 10
toggle bottom row

book
Challenge: Training the Perceptron

Before proceeding with training the perceptron, keep in mind that it uses the binary cross-entropy loss function discussed earlier. The final key concept before implementing backpropagation is the formula for the derivative of this loss function with respect to the output activations, an. Below are the formulas for the loss function and its derivative:

To verify that the perceptron is training correctly, the fit() method also prints the average loss at each epoch. This is calculated by averaging the loss over all training examples in that epoch:

python

Finally, the formulas for computing gradients are as follows:

The sample training data (X_train) along with the corresponding labels (y_train) are stored as NumPy arrays in the utils.py file. Additionally, instances of the activation functions are also defined there:

python
Taak

Swipe to start coding

  1. Compute the following gradients: dz, d_weights, d_biases, and da_prev in the backward() method of the Layer class.
  2. Compute the output of the model in the fit() method of the Perceptron class.
  3. Compute da (dan) before the loop, which is the gradient of the loss with respect to output activations.
  4. Compute da and perform backpropagation in the loop by calling the appropriate method for each of the layers.

If you implemented training correctly, given the learning rate of 0.01, the loss should steadily decrease with each epoch.

Oplossing

Switch to desktopSchakel over naar desktop voor praktijkervaringGa verder vanaf waar je bent met een van de onderstaande opties
Was alles duidelijk?

Hoe kunnen we het verbeteren?

Bedankt voor je feedback!

Sectie 2. Hoofdstuk 10
Switch to desktopSchakel over naar desktop voor praktijkervaringGa verder vanaf waar je bent met een van de onderstaande opties
Onze excuses dat er iets mis is gegaan. Wat is er gebeurd?
some-alt