Active Hackathon

How To Use Genetic Algorithms As A Tool For Feature Selection In Machine Learning

The central idea behind using any feature selection technique is to simplify the models, reduce the training times, avoid the curse of dimensionality without losing much of information.

The popular feature selection methods are:


Sign up for your weekly dose of what's up in emerging technology.
  • Filter method
  • Wrapper method
  • Embedded method

For example, PCAs are popular with dimensionality reduction but the underlying assumptions of PCA depend on linearities. For nonlinear problems which is how real-world scenarios usually are, Genetic algorithms offer significant solutions.

Nature-inspired metaheuristics to use mechanisms like reproduction, mutation, recombination and selection. The solutions to the optimisation problem play the role of individuals in a population and the fitness is used to determine the quality of the solution.

How Do GAs Excel

When the number of features is very large, the GA becomes very computationally demanding and its run time can be prohibitive. To reduce the time required for training, a two-phase hybrid combination of filtering and wrapping is considered.

Chromosome1 (Parent1): [0 0 1 1 0 1 1 0 1]

Chromosome2 (Parent2): [1 1 1 0 0 0 1 1 1]

Two-point crossover operator ↓

Chromosome1 (Child1): [0 0 1 0 0 0 1 0 1]

Chromosome2 (Child2): [1 1 1 1 0 1 1 1 1]

In the first phase, filtering is applied as a pre-processing step and the top-ranked features are selected based on a pre-defined but tunable threshold.

In the second phase, the GA-based wrapper is applied to the selected features, which in practice uses a much smaller search space instead of the original large search space.

This heuristic optimi sation technique may or may not result in combinations of features that have high discriminative power.

Where noise can significantly skew results and dimensionality is very large, genetic algorithms are still capable of optimising a correlated objective function.

Multi dimensional covariance map makes the creation of offspring or desirable feature selection, Bayesian.

While doing GAs it can be observed that the first generation is always larger.

Genetic Algorithms For Finding Galaxies

In this paper, the researchers try to tweak in the classification technique by combining genetic algorithms with support vector machines(SVM).

Indian researcher Sidharth Kumar and his colleagues introduced a mutation in the genes with constant probability. The genes which are chosen to be mutated, are replaced with any of the genes that are not part of either parent, with uniform probability of choosing from the remaining genes.

This allows for genes that are not part of the current gene pool to be expressed. The conceptual simplicity of genetic algorithms combined with their evolutionary analogy as applied to some of the hardest multi-parameter global optimisation problems makes them highly sought after.

For SVM classification, the true positive rate is used as the fitness function. For SVM regression, a custom fitness function based on the problem at hand is chosen.

Once the fitness of all organisms has been evaluated, a new generation of same size as that of the parent generation is created using roulette selection. The GA then runs until it reaches a pre-defined stopping criterion. We use here the posterior distribution of the parameters.

By using Genetic Algorithms to select relevant features, and Support Vector Machines to estimate the quantity of interest using the selected features, the researchers

show that the combination of these two methods yields remarkable results, and offers an interesting opportunity for future large surveys which will gather large amount of data.

In the case of star/galaxy separation, the improvements over existing methods are a consequence of adding more information.

Getting to the target solution in a swarm of data points requires devouring on historical results while stating close to the reality in order to avoid overfitting. Converging on approximate solution is done by various methods of which stochastic optimisation method is one. Probabilistic model building genetic algorithms are a part of stochastic optimisation methods.

These algorithms generate new solutions using an implicit distribution defined by one or more variables. These evolutionary algorithms use an explicit probability distribution encoded by a Bayesian network.

Hyperparameter selection is a key task in improving neural networks and the implicit characteristic of genetic algorithms to implicitly search for best fit strings makes it a suitable contender for machine learning and AI applications as well along with other optimisation problems.

“If you can do feature selection on a large enough set of data, which consists of all independent variables then you can do pretty much anything you want,” says Sidharth Kumar of Flipkart at MLDS 2019, talking about how ubiquitous machine learning is and how he applied his Sidharth Kumar, a Data Scientist at Flipkart says how ubiquitous machine learning is and how he applied this knowledge in observing black holes and conducting analytics for retail.

Read more about his work here

Also watch:

More Great AIM Stories

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM