We implement a basic RNN to demonstrate how the network processes sequential data and makes predictions. A small, dummy dataset is used to showcase how the RNN learns from data and adjusts its weights.

• Creating dummy data: first, we generate a small dataset that consists of simple numerical sequences. These sequences will be used to train our RNN to learn patterns within the data;

sequences = np.random.rand(self.num_samples, self.seq_length, input_size).astype(np.float32)
labels = np.zeros(self.num_samples, dtype=np.int64)

• Building the RNN model: we create a simple RNN architecture with one hidden layer. The input layer receives data, while the hidden layer processes it and passes the output to the next step. The final layer provides the prediction;

self.hidden_size = hidden_size
self.num_layers = num_layers
self.rnn = nn.RNN(input_size, hidden_size, num_layers, batch_first=True)
self.fc = nn.Linear(hidden_size, output_size)

• Training the RNN: during training, the RNN adjusts its internal weights through backpropagation to minimize prediction errors. We will use a loss function (such as mean squared error) to measure the model's performance;

for i, (sequences, labels) in enumerate(dataloader):
    # --- Forward pass ---
    outputs = model(sequences) # Get model predictions (logits)
    loss = criterion(outputs, labels) # Calculate the loss

    # --- Backward pass and optimization ---
    optimizer.zero_grad()   # Clear gradients from the previous iteration
    loss.backward()         # Compute gradients (Backpropagation Through Time happens here)
    optimizer.step()        # Update model parameters based on gradients

    # --- Track metrics ---
    total_loss += loss.item() # Accumulate the loss (loss.item() gets the scalar value)

    # Calculate accuracy for the batch
    _, predicted_classes = torch.max(outputs.data, 1) # Get the index (class) with the highest score
    total_samples += labels.size(0) # Add the number of samples in this batch
    correct_predictions += (predicted_classes == labels).sum().item()

• Evaluating the model: after training, we test the RNN's ability to predict future data points based on the learned patterns from the dummy data. This helps us see how well the model has generalized;

all_sequences, all_labels = dataset[:]
# Get model predictions for the entire dataset
outputs = model(all_sequences)

# Find the predicted class for each sample
_, predicted = torch.max(outputs.data, 1)

# Calculate total number of samples and correct predictions
total = all_labels.size(0)
correct = (predicted == all_labels).sum().item()

# Print the final accuracy
print(f'Accuracy of the model on the {total} training sequences: {100 * correct / total:.2f} %')

• Code example: the code used in this chapter can be downloaded.

In summary, implementing a basic RNN involves preparing data, defining the architecture, training the model, and evaluating its performance. This hands-on approach provides insight into how RNNs can be applied to sequence prediction tasks.