Active Hackathon

Is Deep Learning Necessary For Simple Classification Tasks

Deep learning (DL) models are known for tackling the nonlinearities associated with data, which the traditional estimators such as logistic regression couldn’t. However, there is still a cloud of doubt with regards to the increased use of computationally intensive DL for simple classification tasks.  To find out if DL really outperforms shallow models significantly, the researchers from the University of Pennsylvania experiment with three ML pipelines that involve traditional methods, AutoML and DL in a paper titled, ‘Is Deep Learning Necessary For Simple Classification Tasks.’

How Valid Is The Argument Of AutoML vs DL 

via paper by UPenn

The UPenn researchers stated that a support-vector machine (SVM) model might predict more accurately susceptibility to a certain complex genetic disease than a gradient boosting model trained on the same dataset. Moreover, choosing different hyperparameters within that SVM model can vary performances. These hyperparameter optimisations are usually done with the help of prior experience, brute-force search. This is where AutoML comes into the picture. 

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

AutoML provides methods for automatically selecting these options from a multitude of possible architecture configurations.

For the experiments, the authors consider TPOT (Tree-based Pipeline Automation Tool), a Python-based AutoML tool that uses genetic programming to identify optimal ML pipelines for either regression or classification on a given (labeled) dataset.

The above figure illustrates the model pipelines used in this study where to the left, we can see a pipeline of deep learning strategy with no AutoML. In the middle, we have a standard (no neural networks) TPOT pipeline containing a logistic regression classifier and a kernel SVM classifier. To the right, a TPOT-NN pipeline containing two multilayer perceptron estimators.

The tree based tool, wrote the authors, maintains a balance between high performance and low model complexity.

The three model pipelines were tested on 6 different datasets of hills and valleys, and the results show that neural network estimators had the lowest average prediction accuracy while the TPOT-NN pipelines performed best. In spite of having a neural network in the mix, the TPOT-NN pipelines performed only slightly better than the standard ones.

However, TPOT-NN’s outperformed the LR classifier in most cases. 

Training time, which has been a key differentiator for deep learning approaches, was compared too. The researchers found that the total training time for TPOT pipelines ranged from 4h 22m to 8d 19h 55m, with a mean training time of 1d 0h 49m. To train a single pipeline, on average, there was an increase of 629% for neural network pipelines and 336% for TPOT-NN pipelines versus standard TPOT. 

The parameters for this tree based automation tool included 100 training generations with a population size of 100 in each generation for evaluation. 

The study states that the NN-only models yielded highly inconsistent performance on several datasets. However, this is unsurprising as these datasets contain sequence data, where an estimator must be able to identify a ‘hill’ or a ‘valley’ that could occur at any location in the sequence of 100 features. 

One possible explanation for the large difference in classification accuracy variance between NN-only and TPOT-NN was attributed to the better performance of heuristic and ensemble methods consistently.

Based on the results the authors conclude that AutoML (and TPOT-NN, in particular) maybe useful for discovering new neural network “motifs” to be composed into larger networks. And, by repeating the internal architecture to a final depth of 152 hidden layers, the result is virtually identical to the version of ResNet.

Though DL on its own might have been outperformed by the AutoML tool, the authors recommend an integrated approach towards AutoML and DL rather than their divided usage. Furthermore, they stated that the addition of multilayer perceptron classifiers into the pool of available operators improves the performance of AutoML.

Link to the paper here.

More Great AIM Stories

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MOST POPULAR

The curious case of Google Cloud revenue

Porat had earlier said that Google Cloud was putting in money to make more money, but even with the bucket-loads of money that it was making, profitability was still elusive.

Global Parliaments can do much more with Artificial Intelligence

The world is using AI to enhance the performance of its policymakers. India, too, has launched its own machine learning system NeVA, which at the moment is not fully implemented across the nation. How can we learn and adopt from the advancement in the Parliaments around the world? 

Why IISc wins?

IISc was selected as the world’s top research university, trumping some of the top Ivy League colleges in the QS World University Rankings 2022

[class^="wpforms-"]
[class^="wpforms-"]