Key takeaways
- Generative Adversarial Networks (GANs) play a crucial role in the commercialisation of AI across various industries.
- AI can now create art that fetches high prices, demonstrating its growing capabilities.
- The concept of GANs has evolved significantly since their introduction in 2014, leading to diverse applications.
- GANs are instrumental in generating realistic images, music, and aiding in medical anomaly detection.

The Working Principle
GANs are generative models devised by Goodfellow et al. in 2014. They work on the principle of generating and discriminating the inputs. The two networks, Generator and Discriminator go toe to toe with each other like arch-nemesis; benefitting the overall model eventually.
Generator Architecture
Source: researchgate
The Generator network has 4 convolutional layers, all followed by BatchNorm (except for the output layer) and Rectified Linear unit (ReLU) activations.
Rectified Linear Unit or ReLU is now one of the most widely used activation functions. The function operates on max(0,x), which means that anything less than zero will be returned as 0 and linear with the slope of 1 when the value is greater than 0.
The network takes as an input drawn from a normal distribution and is fed through the consecutive layers. Each of these layers represent a convolution operation.
Discriminator Architecture
Source: researchgate
The discriminator is also a 4 layer CNN with BatchNorm (except its input layer) and leaky ReLU activations.
The drawback with ReLU function is their fragility, that is, when a large gradient is made to flow through ReLU neuron, it can render the neuron useless and make it unable to fire on any other datapoint again for the rest of the process. In order to address this problem, leaky ReLU were introduced. So, unlike in ReLU when anything less than zero is returned as zero, leaky version instead has a small negative slope, which is crucial for the functioning of a Discriminator network.
Half of the time the Discriminator network receives images from the training set and the other half from the generator.
The Discriminator has to output probabilities close to 1 for real images and near 0 for fake images. To do that, the discriminator takes sum of two partial losses- One for maximizing the probabilities for the real images and another for minimizing the probability of fake images.
As training progresses, the generator starts to output images that look closer to the images from the training set. That happens because the generator trains to learn the data distribution that composes the training set images.
At the same time, the discriminator starts to get real good at classifying samples as real or fake.
Summary
The whole concept of GANs can be summarised as, “the generator is trying to fool the discriminator while the discriminator is trying to not get fooled by the generator. As the models train through alternating optimization, both methods are improved until a point where the counterfeits are indistinguishable from the genuine ones.”
Further reading:
https://adeshpande3.github.io/Deep-Learning-Research-Review-Week-1-Generative-Adversarial-Nets https://skymind.ai/wiki/generative-adversarial-network-gan https://www.freecodecamp.org/news/an-intuitive-introduction-to-generative-adversarial-networks-gans-7a2264a81394/Spotted an error? Tell our editors. Read our editorial standards and corrections.




