Advertisement

Active Hackathon

This New Tool Helps Developers In Effective Hyperparameter Tuning 

Machine learning models are getting complicated and often have many hyperparameters. In order to ease the exhaustive trial and error kind of experimental procedures, Facebook AI has released HiPlot for the public. HiPlot had been used by the developers at Facebook AI to explore and efficiently analyse hyperparameter tuning of deep neural networks with dozens of hyperparameters and more than 100,000 experiments. 

This new tool helps practitioners to visualise the influence of hyperparameters on a certain task. Machine learning developers can play with this tool by dragging and sliding the cursor and see the changes in real-time. This is something similar to an equaliser in music player settings on our phones.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

How To Use HiPlot

via Facebook AI

HiPlot uses parallel plots and other graphical ways to represent information more clearly, and it can run quickly on a Jupyter notebook with no setup required. 

Instructions on how to use HiPlot:

  • On the parallel plot, each line represents one data point. Slicing on the valid ppl axis reveals that higher values for learning rate (lr) lead to better models.
  • Un-slice the valid ppl axis by clicking on the axis, but outside of the current slice. Slice on the lr axis values above 1e-2, and then click the ‘Keep’ button.
  • Check how training behaves with a right-click on the epoch axis title and selecting ‘Set as X-axis’. Similarly, set valid ppl as the ‘Y-axis’. Once you have done both, an XY line plot should appear below the parallel plot.
  • Slicing through the dropout, embedding_size and lr axis reveals how they can affect the training dynamics — convergence speed and maximum performance.

In short, the whole point of using a HiPlot is to get insights about hyperparameters without going knee-deep into the models. By slicing through the “loss” axis, one can make observations such as the influence of larger learning rates led to better performance.

Installation:

>> pip install hiplot

Sample execution:

  • Step 1: Use HiPlot on one of the models, and get the hyperparameter values.
  • Step 2: Now play with the cursor on HiPlot, and get insights
  • Step 3: Use these insights to update the model.

import hiplot as hip

data = [{‘dropout’:0.1, ‘lr’: 0.001, ‘loss’: 10.0, ‘optimizer’: ‘SGD’}]

hip.Experiment.from_iterable(data).display()

Here are the results:

Now when the top two drop-out values are selected:

We can see above how a change in the dropout values influences the learning rate just by looking at the plot. Now drag the cursor to whatever hyperparameter you want to check in order to get an idea of whether to go with certain values or not.

There has been a flurry of interactive tools being released for developers. Last year we saw the rise in popularity of TensorBoard where one can visualise data in the most intuitive way.

Similar to HiPlot, Tensorflow’s Tensorboard too offers a visualisation tool for hyperparameter tuning. It uses parallel coordinates like HiPlot. 

However, TensorBoard has much more to offer than HiPlot. HiPlot being lightweight and straight forward, has been good for those who want to have a quick look into what is happening behind the training.

Apart from interactive tools like HiPlot and Tensorboard, there are conventional approaches like Hyperparameter hunter. Hyperparameter Hunter was released last year, which allowed the users to run all of the benchmark and one-off experiments through it, and it also doesn’t start optimisation from scratch like other libraries. It considers all the previously run experiments and previous optimisation rounds that have been already run through it. 

Typically, the hyperparameter exploration process is painstakingly manual, given that the search space is vast, and the evaluation of each configuration can be expensive; interactive tools are a gift to the machine learning practitioners.

Try HiPlot here.

More Great AIM Stories

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MOST POPULAR
How Data Science Can Help Overcome The Global Chip Shortage

China-Taiwan standoff might increase Global chip shortage

After Nancy Pelosi’s visit to Taiwan, Chinese aircraft are violating Taiwan’s airspace. The escalation made TSMC’s chairman go public and threaten the world with consequences. Can this move by China fuel a global chip shortage?

Another bill bites the dust

The Bill had faced heavy criticism from different stakeholders -citizens, tech firms, political parties since its inception

So long, Spotify

‘TikTok Music’ is set to take over the online streaming space, but there exists an app that has silently established itself in the Indian market.

[class^="wpforms-"]
[class^="wpforms-"]