What Are the Benefits of Generative Models over Discriminative Models in Deep Learning?

The study of Machine Learning (ML) and Deep Learning (DL) is an incredibly captivating and rapidly evolving field of research in the 21st century. Through the utilisation of these breakthroughs, robots are now able to ingest past data and make decisions based on their predictions of what may transpire in the future.

The concept of leveraging the human mind and past experiences to inform our current and future decisions is the foundation for machine learning (ML) and deep learning (DL). While these technologies have already been applied in a variety of ways, their potential for transforming our world is virtually limitless.

Given that computers cannot comprehend data and information in the same way as humans do, they depend on mathematics, algorithms, and data pipelines to derive insights from raw data. To increase the machine’s productivity, two approaches can be taken: increasing the amount of data available and/or improving the accuracy of algorithms.

It is estimated that billions of pieces of data are generated each day across the world, providing an abundance of information to be accessed. To effectively utilise these vast amounts of data, it is essential that we develop new algorithms or increase the capacity of existing algorithms.

Algorithms and models are constructed on a basis of mathematics, which typically includes calculus, probability, statistics, and other related disciplines. These fall into two distinct categories that can be easily identified and distinguished from each other.

  1. Models with discriminatory power
  2. Models with generative potential

Both frameworks will be discussed in this essay. We’ll start by reading a short narrative to get inside their heads.

The top siblings

Zed and Zack are identical twins who are incredibly close; it is difficult to tell them apart as they look identical. Both of these siblings are incredibly intelligent, and they have risen to the top of their graduating class. Zed is remarkable in that he has the ability to master any subject with enough time and focus. He is incredibly dedicated in his studies, immersing himself in a topic until he is able to understand it completely. Despite the fact that a great deal of work is required to achieve this level of understanding, Zed is often pressed for time when it comes to taking tests. This is especially true when compared to his brother, who usually has more time to prepare.

Zack has adopted a technique of creating a mental map to arrange his study materials. Initially, he comprehends the overall concept of a subject, and then he is able to discern and recognise the patterns in the minutiae. Through this approach, he is able to gain a better understanding of the subject matter and develop a greater degree of flexibility in his thought processes. In other words, he is able to “learn” better by recognising and understanding the distinctions.

They’ve maintained their place at the top of their class despite the fact that the brothers have quite different methods to studying.

According to this comparison, generative models are similar to Zed while discriminative models are similar to Zack.

Following the generation of estimated parameters from the data and the application of Bayes’ theorem to calculate the posterior probability of P(Y|X), generative classifiers assume a functional form for both P(Y) and P(X|Y). In contrast, discriminative classifiers directly estimate the parameters from the given data whilst also assuming a functional form of P(Y|X).

Model that makes distinctions

Discriminative models, also referred to as conditional models, are extensively utilised in supervised machine learning. These models are capable of effectively classifying data points into distinct groups and utilising probability estimates and maximum likelihood to learn the boundaries between those groups. The models are highly effective in achieving their intended purpose.

These models are very immune to outliers. While they are preferable than generative models, they may still have significant misclassification issues.

Some of the most common types of discriminative models, along with short descriptions of each, are as follows:

  1. Regression Analysis Using Logit model: Linear regression and logistic regression are two forms of categorization models that can be compared in terms of their uses. Linear regression is used to predict a continuous dependent variable, while logistic regression is used for distinguishing between two or more classes. Both models are useful for different purposes, and by understanding the differences between them, one can decide which model is the most suitable for a particular situation.
  2. AI that uses SVMs, or support vector machines: In both regression and classification tasks, the support vector machine (SVM) approach is a powerful and reliable learning technique. SVMs are used to categorise points in an n-dimensional data space by employing decision boundaries, with the optimal boundary being a hyperplane.
  3. Determination trees: Choices and their potential repercussions are mapped in a tree-like graphical form. It’s like a more powerful form of “If-then” statements.

Popular examples include neural networks, k-nearest neighbours, conditional random fields, random forests, etc.

Model that generates

Generative models can be used to produce new data points, as the name implies. These models are often the most suitable choice for machine learning tasks that do not involve direct human supervision.

Instead of only modelling the cutoff for each class, generative models represent the whole distribution of data and learn the various data points.

The primary disadvantage of generative models is that they are not as adept at dealing with outliers as discriminative models. This approach to machine learning is distinct in that it is indirect; instead of directly predicting the output, it estimates the prior probability P(Y) and the likelihood probability P(X|Y) to arrive at P(Y|X). The mathematics behind generative models are also intuitive and easy to understand.

By plugging them into Bayes’ theorem’s formula, we may get a precise estimate of P(Y|X).

The examples and explanation of generative models follow.

  1. “””Bayesian Network” This model, which is commonly referred to as a Bayes’ network, makes use of a directed acyclic graph (DAG) to conduct Bayesian inference regarding a set of random variables, thus allowing for the determination of the probability of certain outcomes. It has multiple applications, such as prediction, anomaly detection, and time series prediction.
  2. Modelling Using autoregression: Time series modelling is where it really shines, since it uses data from the past to make predictions about the future.
  3. To generate an adversarial network (GAN): This system utilises two submodels that are based on deep learning technology. The discriminative model is trained to recognise the differences between authentic and counterfeit data points generated by the generator model.

Naive Bayes, the Markov random field, the hidden Markov model (HMM), the latent Dirichlet allocation (LDA), etc. are some more examples.

Which task better suits Deep Learning: discriminative or generative?

Generative models are capable of understanding how data is distributed within a space, while discriminative models are able to identify the boundaries between different classes. These two approaches are distinct from each other, making them suitable for a variety of purposes.

Artificial Neural Networks (ANNs), Convolutional Neural Networks (CNNs), and Recurrent Neural Networks (RNNs) are all examples of supervised machine learning techniques that are used in Deep Learning. ANNs are the first of the three methods to employ artificial neurons, backpropagation, weights, and biases to recognise patterns from inputs. CNNs are commonly used for computer vision and image recognition applications, producing effects by combining pertinent elements of an image. RNNs, the most recent of the three, are employed in advanced applications such as natural language processing, handwriting recognition, and time series analysis.

As deep learning excels at supervised tasks, these are the domains where discriminative models shine.

In addition to this, deep learning and neural networks can be used to group together photographs based on their similarities. Commonly used unsupervised deep learning techniques include autoencoders, Boltzmann machines, and self-organising maps. Generative models can be applied to a variety of tasks, such as exploratory data analysis (EDA) of large datasets, noise reduction in images, image compression, anomaly detection, and even the creation of new images.

The fascinating website This Person Does Not Exist – Random Face Generator utilises a generative model called StyleGAN to create highly realistic human faces, even though the people depicted in the photographs are not real.

Join the Top 1% of Remote Developers and Designers

Works connects the top 1% of remote developers and designers with the leading brands and startups around the world. We focus on sophisticated, challenging tier-one projects which require highly skilled talent and problem solvers.
seasoned project manager reviewing remote software engineer's progress on software development project, hired from Works blog.join_marketplace.your_wayexperienced remote UI / UX designer working remotely at home while working on UI / UX & product design projects on Works blog.join_marketplace.freelance_jobs