Machine learning models can be classified into discriminative and generative models. In simple words, a discriminative model makes predictions based on conditional probability and is either used for classification or regression. On the other hand, a generative model revolves around the distribution of a dataset to return a probability for a given example.
Discriminative Model
The discriminative model is used particularly for supervised machine learning. Also called a conditional model, it learns the boundaries between classes or labels in a dataset. It creates new instances using probability estimates and maximum likelihood. However, they are not capable of generating new data points. The ultimate goal of discriminative models is to separate one class from another.
Types of discriminative models in machine learning include:
Logistic regression: It is one of the most popular machine learning algorithms. Logistic regression is a mathematical model in statistics that uses previous data to estimate an event’s probability. The output is a categorical or a discrete value. In principle, logistic regression is similar to linear regression. However, there is a small difference between them. While linear regression is used for regression problems, logistic regression is used for classification problems.
Support vector machine: It is a supervised learning algorithm used both for classification and regression problems. A type of discriminative modelling, support vector machine (SVM) creates a decision boundary to segregate n-dimensional space into classes. The best decision boundary is called a hyperplane created by choosing the extreme points called the support vectors.
Decision tree: A type of supervised machine learning model where data is continuously split according to certain parameters. It has two main entities–decision nodes and leaves. While leaves are the final outcomes or decisions, nodes are the points where data is split.
Random forest: It is a flexible and easy-to-use machine learning algorithm that gives great results without even using hyper-parameter tuning. Because of its simplicity and diversity, it is one of the most used algorithms for both classification and regression tasks.
Generative models
Generative models are a class of statistical models that generate new data instances. These models are used in unsupervised machine learning to perform tasks such as probability and likelihood estimation, modelling data points, and distinguishing between classes using these probabilities. Generative models rely on the Bayes theorem to find the joint probability. Common examples of generative models are:
Latent Dirichlet Allocation (LDA): It is a generative probabilistic model with collection of discrete data, each of which is modelled as a finite mixture. Some of the common applications of LDA are collaborative filtering and content-based image retrieval.
Bayesian Network: Also known as Bayes network, it is a generative probabilistic graphical model that gives an efficient representation of joint probability distribution over a set of random variables.
Hidden Markov model: It is a statistical model known for its effectiveness in modelling the correlation between adjacent symbols, events or domains, finding major application in speech recognition and digital communication.
Autoregressive model: An AR model predicts future values based on past values. This kind of model is good at handling a wide range of time-series patterns.
Generative adversarial network: GANs have gained much popularity recently. A GAN model has two parts–generator and discriminator. The generative model captures the data distribution and the discriminative model estimates the probability of sample coming from training data rather than the generative model.
Discriminative Vs Generative Models
Discriminative models draw boundaries in the data space, while generative ones model how data is placed throughout the space. Mathematically speaking, a discriminative machine learning trains a model by learning parameters that maximise the conditional probability P(Y|X), but a generative model learns parameters by maximising the joint probability P(X,Y).
Because of their different approaches to machine learning, both are suited for specific tasks. Generative models are useful for unsupervised learning tasks and discriminative models work better for supervised learning tasks.
