Autoencoders Architecture

An autoencoder is an artificial neural network used for unsupervised learning, particularly in data compression and feature learning.
It comprises an encoder and a decoder network, where the encoder compresses the input data into a lower-dimensional latent space representation, and the decoder reconstructs the original input data from this representation.

Note

We will consider the latent space of the autoencoder in the next chapter in more detail!

Training Process

The training process for an autoencoder is similar to training a simple Multilayer Perceptron (MLP). Here's how it works:

1. Forward Pass: the input data (which is the image in this case) is fed into the network. It travels through the layers, undergoing transformations at each layer;
2. Output Generation: the network produces an output, which is a reconstructed version of the original input image;
3. Error Calculation: the difference between the original image and the reconstructed image is calculated. This difference is called the reconstruction error;
4. Backpropagation: the reconstruction error is then propagated backward through the network. This process helps the network adjust the weights of its connections to minimize the error in future reconstructions;
5. Training Stop: the training process continues until the reconstruction error reaches an acceptable minimum value.

Key Difference: unlike training a classifier where we use classification metrics (like accuracy), in autoencoders, we directly compare the reconstructed image with the original image to calculate the error.

Note

Autoencoders cannot be directly employed to generate new images. However, they serve as the foundation for the development of Variational Autoencoders (VAEs) and Conditional Variational Autoencoders (CVAEs), which are capable of generating images.