GANs

A Generative Adversarial Network (GAN) utilizes an iterative training process involving two competing neural networks: a generator and a discriminator. This adversarial training fosters continuous improvement in both networks.

Network Roles

1. Generator: this network functions as a data synthesizer, aiming to create novel data instances indistinguishable from real data;
2. Discriminator: this network acts as a data verifier, meticulously evaluating the presented data and attempting to classify it as authentic (real data) or synthetic (generated by the generator).

Training Principle

You may notice that this architecture differs from what we used in previous chapters. Now, we have two neural networks, and it may need to be clarified how to provide a training process.

The Iterative Training Loop:

1. Data Generation: the generator starts by creating a batch of new data;
2. Data Presentation: the generator shows this new data to the discriminator for evaluation;
3. Data Classification: the discriminator carefully examines each data point and decides if it is real or synthetic. Ideally, the discriminator needs help telling real and generated data apart;
4. Network Updates: Based on the discriminator's feedback, both networks adjust their parameters to improve their performance:
   • Generator Update: the generator adjusts its internal parameters based on how well it fooled the discriminator. This helps it create more realistic data in future iterations;
   • Discriminator Update: the discriminator updates its parameters to better distinguish real data from generated data if the generator fooled it.

Separate Losses for Generator and Discriminator

The ultimate goal is to train the generator to create realistic and diverse images that the discriminator mistakes for real.

1. Generator Loss: this measures how well the generated images deceive the discriminator. It encourages the generator to create outputs the discriminator wrongly classifies as real;
2. Discriminator Loss: this measures the discriminator's ability to tell real images from generated ones. It includes two parts:
   • Loss from Real Images: penalizes the discriminator for wrongly classifying real images as fake;
   • Loss from Generated Images: penalizes the discriminator for wrongly classifying generated images as real.