Machine learning models are getting complicated and often have many hyperparameters. In order to ease the exhaustive trial and error kind of experimental procedures, Facebook AI has released HiPlot for the public. HiPlot had been used by the developers at Facebook AI to explore and efficiently analyse hyperparameter tuning of deep neural networks with dozens of hyperparameters and more than 100,000 experiments.
This new tool helps practitioners to visualise the influence of hyperparameters on a certain task. Machine learning developers can play with this tool by dragging and sliding the cursor and see the changes in real-time. This is something similar to an equaliser in music player settings on our phones.
How To Use HiPlot
via Facebook AI
HiPlot uses parallel plots and other graphical ways to represent information more clearly, and it can run quickly on a Jupyter notebook with no setup required.
Instructions on how to use HiPlot:
- On the parallel plot, each line represents one data point. Slicing on the valid ppl axis reveals that higher values for learning rate (lr) lead to better models.
- Un-slice the valid ppl axis by clicking on the axis, but outside of the current slice. Slice on the lr axis values above 1e-2, and then click the ‘Keep’ button.
- Check how training behaves with a right-click on the epoch axis title and selecting ‘Set as X-axis’. Similarly, set valid ppl as the ‘Y-axis’. Once you have done both, an XY line plot should appear below the parallel plot.
- Slicing through the dropout, embedding_size and lr axis reveals how they can affect the training dynamics — convergence speed and maximum performance.
In short, the whole point of using a HiPlot is to get insights about hyperparameters without going knee-deep into the models. By slicing through the “loss” axis, one can make observations such as the influence of larger learning rates led to better performance.
Installation:
>> pip install hiplot
Sample execution:
- Step 1: Use HiPlot on one of the models, and get the hyperparameter values.
- Step 2: Now play with the cursor on HiPlot, and get insights
- Step 3: Use these insights to update the model.
import hiplot as hip
data = [{‘dropout’:0.1, ‘lr’: 0.001, ‘loss’: 10.0, ‘optimizer’: ‘SGD’}]
hip.Experiment.from_iterable(data).display()
Here are the results:
Now when the top two drop-out values are selected:
We can see above how a change in the dropout values influences the learning rate just by looking at the plot. Now drag the cursor to whatever hyperparameter you want to check in order to get an idea of whether to go with certain values or not.
There has been a flurry of interactive tools being released for developers. Last year we saw the rise in popularity of TensorBoard where one can visualise data in the most intuitive way.
Similar to HiPlot, Tensorflow’s Tensorboard too offers a visualisation tool for hyperparameter tuning. It uses parallel coordinates like HiPlot.
However, TensorBoard has much more to offer than HiPlot. HiPlot being lightweight and straight forward, has been good for those who want to have a quick look into what is happening behind the training.
Apart from interactive tools like HiPlot and Tensorboard, there are conventional approaches like Hyperparameter hunter. Hyperparameter Hunter was released last year, which allowed the users to run all of the benchmark and one-off experiments through it, and it also doesn’t start optimisation from scratch like other libraries. It considers all the previously run experiments and previous optimisation rounds that have been already run through it.
Typically, the hyperparameter exploration process is painstakingly manual, given that the search space is vast, and the evaluation of each configuration can be expensive; interactive tools are a gift to the machine learning practitioners.
Try HiPlot here.
