The central idea behind using any feature selection technique is to simplify the models, reduce the training times, avoid the curse of dimensionality without losing much of information.
The popular feature selection methods are:
- Filter method
- Wrapper method
- Embedded method
For example, PCAs are popular with dimensionality reduction but the underlying assumptions of PCA depend on linearities. For nonlinear problems which is how real-world scenarios usually are, Genetic algorithms offer significant solutions.
Nature-inspired metaheuristics to use mechanisms like reproduction, mutation, recombination and selection. The solutions to the optimisation problem play the role of individuals in a population and the fitness is used to determine the quality of the solution.
How Do GAs Excel
When the number of features is very large, the GA becomes very computationally demanding and its run time can be prohibitive. To reduce the time required for training, a two-phase hybrid combination of filtering and wrapping is considered.
Chromosome1 (Parent1): [0 0 1 1 0 1 1 0 1]
Chromosome2 (Parent2): [1 1 1 0 0 0 1 1 1]
Two-point crossover operator ↓
Chromosome1 (Child1): [0 0 1 0 0 0 1 0 1]
Chromosome2 (Child2): [1 1 1 1 0 1 1 1 1]
In the first phase, filtering is applied as a pre-processing step and the top-ranked features are selected based on a pre-defined but tunable threshold.
In the second phase, the GA-based wrapper is applied to the selected features, which in practice uses a much smaller search space instead of the original large search space.
This heuristic optimi sation technique may or may not result in combinations of features that have high discriminative power.
Where noise can significantly skew results and dimensionality is very large, genetic algorithms are still capable of optimising a correlated objective function.
Multi dimensional covariance map makes the creation of offspring or desirable feature selection, Bayesian.
While doing GAs it can be observed that the first generation is always larger.
Genetic Algorithms For Finding Galaxies
In this paper, the researchers try to tweak in the classification technique by combining genetic algorithms with support vector machines(SVM).
Indian researcher Sidharth Kumar and his colleagues introduced a mutation in the genes with constant probability. The genes which are chosen to be mutated, are replaced with any of the genes that are not part of either parent, with uniform probability of choosing from the remaining genes.
This allows for genes that are not part of the current gene pool to be expressed. The conceptual simplicity of genetic algorithms combined with their evolutionary analogy as applied to some of the hardest multi-parameter global optimisation problems makes them highly sought after.
For SVM classification, the true positive rate is used as the fitness function. For SVM regression, a custom fitness function based on the problem at hand is chosen.
Once the fitness of all organisms has been evaluated, a new generation of same size as that of the parent generation is created using roulette selection. The GA then runs until it reaches a pre-defined stopping criterion. We use here the posterior distribution of the parameters.
By using Genetic Algorithms to select relevant features, and Support Vector Machines to estimate the quantity of interest using the selected features, the researchers
show that the combination of these two methods yields remarkable results, and offers an interesting opportunity for future large surveys which will gather large amount of data.
In the case of star/galaxy separation, the improvements over existing methods are a consequence of adding more information.
Getting to the target solution in a swarm of data points requires devouring on historical results while stating close to the reality in order to avoid overfitting. Converging on approximate solution is done by various methods of which stochastic optimisation method is one. Probabilistic model building genetic algorithms are a part of stochastic optimisation methods.
These algorithms generate new solutions using an implicit distribution defined by one or more variables. These evolutionary algorithms use an explicit probability distribution encoded by a Bayesian network.
Hyperparameter selection is a key task in improving neural networks and the implicit characteristic of genetic algorithms to implicitly search for best fit strings makes it a suitable contender for machine learning and AI applications as well along with other optimisation problems.
“If you can do feature selection on a large enough set of data, which consists of all independent variables then you can do pretty much anything you want,” says Sidharth Kumar of Flipkart at MLDS 2019, talking about how ubiquitous machine learning is and how he applied his Sidharth Kumar, a Data Scientist at Flipkart says how ubiquitous machine learning is and how he applied this knowledge in observing black holes and conducting analytics for retail.
Read more about his work here
Also watch:
