Summary  
This chapter explains how the sigmoid and tanh activation functions map inputs into fixed ranges—[0,1] for sigmoid and [–1,1] for tanh—and how these ranges are used as gating mechanisms in recurrent neural networks to control information retention and state updates.  

General domain of usage  
Recurrent neural networks

The **sigmoid** and **tanh** activation functions are explored, as they play a crucial role in the functioning of **RNNs**. 

The **sigmoid** and **tanh** functions transform inputs into outputs, enabling the model to make predictions.


Definition

- **Sigmoid activation**: the sigmoid function maps input values to an output range between 0 and 1. It is commonly used in binary classification tasks, as its output can be interpreted as a probability. However, it suffers from the **vanishing gradient problem** when the input values are very large or very small;  
- **Tanh activation**: the **tanh** function is similar to the sigmoid but maps the input values to an output range between -1 and 1. It helps center the data around zero, which can aid learning. Despite its benefits, it also suffers from the vanishing gradient problem in certain situations;  
- **Working of sigmoid and tanh**: both functions work by squashing the input values into a bounded range. The primary difference lies in their output range: **sigmoid** (0 to 1) vs. **tanh** (-1 to 1), which affects how the network processes and updates the information.

In the next chapter, we will look at how these activation functions play a role in **LSTM** networks and how they help overcome some of the limitations of standard RNNs.


What is the output range of the sigmoid activation function?

Focus exclusively on Recurrent Neural Networks (RNNs) and their application to sequence generation. This advanced module covers the core theory of RNNs and practical approaches to modeling sequential data, preparing you for hands-on work with LSTM and GRU networks in Python.

Sigmoid and Tanh Activations for RNNs