Courses /

Introduction to Neural Networks

1. Concept of Neural Network

First, we will discuss what a neural network is and how it works. And also consider the scope of its application.

What is a Neural Network?

Neural Network Structure

What is a Neuron?

Activation Functions

Forward and Backward Propagation

Applications of Deep Learning in the Real World

2. Neural Network from Scratch

Next, we will try to build our own neural network and see how efficiently it copes with learning. We will also consider a ready-made solution from the scikit-learn library.

Single Neuron Implementation

Model Training and Evaluation

Neural Network with scikit-learn

3. Conclusion

Finally, we will give you some additional useful information on how to understand which model to use and what types of neural networks there are. To complete the course, you will be tested on your acquired knowledge.

Neural Networks or Traditional Models

Other types of Neural Networks

Manual Hyperparameter Tuning

Automatic Hyperparameter Tuning

Final Quiz

Summary

Automatic Hyperparameter Tuning

Task

Rather than manually selecting specific values for our model's hyperparameters, employing Random Search (RandomizedSearchCV) can be a more efficient strategy to discover the optimal settings. The concept is somewhat akin to GridSearchCV, yet it comes with a significant difference.

In the realm of neural networks, exhaustively searching through every possible combination of parameters, as GridSearchCV does, can be prohibitively time-consuming.

This is where Random Search shines. Instead of evaluating all parameter combinations, it samples a random subset of them, which often leads to faster and surprisingly effective results.

Here is an example of Random Search usage:

Your task is:

Generate values for two hidden layers with number of neurons in range from 20 to 30 with step 2.
Set the values for the learning rate to choose from. As we saw in the previous chapter, the model performs well with a learning rate of around 0.01. So we can reduce the search area to the values 0.02, 0.01, and 0.005.
Generate 10 random values for epochs in range from 10 to 50.
Apply random search for 4 models (iterations).
Evaluate the model.

Everything was clear?

Section 3. Chapter 4

Course Content

Introduction to Neural Networks

1. Concept of Neural Network

What is a Neural Network?Neural Network Structure What is a Neuron?Activation Functions Forward and Backward Propagation Applications of Deep Learning in the Real World

2. Neural Network from Scratch

Single Neuron Implementation Perceptron Layers Forward Propagation Backward Propagation Data Preprocessing Model Training and Evaluation Neural Network with scikit-learn

3. Conclusion

Neural Networks or Traditional Models Other types of Neural Networks Manual Hyperparameter Tuning Automatic Hyperparameter Tuning Final Quiz Summary

Introduction to Neural Networks

Automatic Hyperparameter Tuning

Task

In the realm of neural networks, exhaustively searching through every possible combination of parameters, as GridSearchCV does, can be prohibitively time-consuming.

This is where Random Search shines. Instead of evaluating all parameter combinations, it samples a random subset of them, which often leads to faster and surprisingly effective results.

Here is an example of Random Search usage:

Your task is:

Generate values for two hidden layers with number of neurons in range from 20 to 30 with step 2.
Set the values for the learning rate to choose from. As we saw in the previous chapter, the model performs well with a learning rate of around 0.01. So we can reduce the search area to the values 0.02, 0.01, and 0.005.
Generate 10 random values for epochs in range from 10 to 50.
Apply random search for 4 models (iterations).
Evaluate the model.

Everything was clear?

Section 3. Chapter 4