Alibaba open-sources KNAS, a low consumption AutoML algorithm

KNAS uses a gradient kernel as a proxy for model quality and consumes less computing resources compared to standard techniques.

Researchers at Alibaba Group and Peking University have open-sourced an efficient AutoML algorithm called Kernel Neural Architecture Search. The study looked for a green NAS (Neural Architecture Search) solution that evaluates architectures without training.

KNAS uses a gradient kernel as a proxy for model quality and consumes less computing resources compared to standard techniques. The team proposed the hypothesis: “Gradients can be used as a coarse-grained proxy of downstream training to evaluate randomly-initialized architectures.” The researchers found a gradient kernel (the mean of the Gram matrix (MGM) of gradients) has a strong correlation with a model’s accuracy. The KNAS algorithm computes the MGM for each proposed model architecture, keeping only the best few, calculating the model accuracy for those candidates, and selecting the model with the highest accuracy as the final result.

Usually, neural architecture search systems are used to find the best deep-learning model architecture for a task. The system does this by finding an architecture that is well-suited to deliver the best performance metric on the given task dataset and search space of possible architectures. But, this method demands training each proposed model completely on the dataset, resulting in longer training times.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

More Great AIM Stories

SharathKumar Nair
Sharath is an ardent believer in the ‘Transhumanism’ movement. Anything and everything about technology excites him. At Analytics India Magazine, he writes about artificial intelligence, cybersecurity and the impact these emerging technologies have on day-to-day human lives. When not working on a story, he spends his time reading tech novels and watching sci-fi movies and series.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

Conference, in-person (Bangalore)
Cypher 2023
20-22nd Sep, 2023

3 Ways to Join our Community

Whatsapp group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our newsletter

Get the latest updates from AIM