Visualization helps us understand big data with ease. It helps us identify patterns and get deeper insights or at least make the process easier.\n\n\n\nIn the machine learning and data science spectrum, we often emphasise the importance of visualisations. Visualisations help in uncovering the truth by helping us understand important facts and figures.\n\n\n\nTensorBoard is a tool from Tensorflow that helps in understanding a neural network through interactive graphs and statistics.\n\n\n\nIn this tutorial, we are just interested in HyperParameter tuning which by itself is a great deal in Machine learning.\n\n\n\nSo let's begin:\n\n\n\nData Set\n\n\n\nWe will use data from MachineHack's Predicting The Costs Of Used Cars Hackathon. To download the free dataset, just head to MachineHack and sign up.\n\n\n\nFEATURES:\n\n\n\nName: The brand and model of the car.Location: The location in which the car is being sold or is available for purchase.Year: The year or edition of the model.Kilometers_Driven: The total kilometres driven in the car by the previous owner(s) in KM.Fuel_Type: The type of fuel used by the car.Transmission: The type of transmission used by the car.Owner_Type: Whether the ownership is Firsthand, Second hand or other.Mileage: The standard mileage offered by the car company in kmpl or km\/kgEngine: The displacement volume of the engine in cc.Power: The maximum power of the engine in bhp.Seats: The number of seats in the car.New_Price: The price of a new car of the same model.Price: The price of the used car in INR Lakhs.\n\n\n\nWe will train a neural network to predict the price of a used car based on the above list of features.\n\n\n\nThe data preparation and preprocessing part has already been taken care of and you can find it in the complete code at the end of this article. \n\n\n\nHyperparameter Tuning With TensorBoard\n\n\n\nLet us assume that we have an initial Keras sequential model for the given problem as follows:\n\n\n\n\n\nHere we have an input layer with 26 nodes, a hidden layer with 100 nodes and relu activation function, a dropout layer with a dropout fraction of 0.2, an output layer with a single node for regression and an Adam optimizer.\n\n\n\nNote:\n\n\n\nThe input shape is 26 since after preprocessing the dataset there are 26 independent features.\n\n\n\nWe notice that there are a few hyperparameters that we randomly initialized such as the number of nodes in the hidden layer, the dropout ratio and the optimizer function.\n\n\n\nThere are a number of possible values we can assign for each of these parameters and the effect of these parameters can only be assessed while training the model. Hence trying out different values and combinations for each of the parameters is not a feasible solution.\n\n\n\nWe will use the power of TensorBoard to visualize the performance of the network for each of the different parameters and all in one go.\u00a0\n\n\n\nNote:\n\n\n\nThe following example was done on Google Colab with Tensorflow 2.0.\n\n\n\nSo let's begin!\n\n\n\nImporting Tensorboard Plugin\n\n\n\nfrom tensorboard.plugins.hparams import api as hp\n\n\n\nWe will start by importing the hparams plugin available in the tensorboard.plugin module.\n\n\n\nInitializing HyperParameters \n\n\n\n\n\n\n\nIn the above code block, we initialize values for the hyperparameters that need to be assessed. We then set the metrics of the model to RMSE. Since Tensorboard works with log files that are created during the training process we create logs for the training process that records the losses, metrics and other measures during training.\n\n\n\nA function To Train And Validate\n\n\n\nNow we create a function to train and validate the model which will take the hyperparameters as arguments. Each combination of hyperparameters will run for 6 epochs and the hyperparameters are provided in an hparams dictionary and used throughout the training function.\n\n\n\n\n\n\n\nA function to log the training process\n\n\n\nThe following function will initiate the training process with the hyperparameters to be assessed and will create a summary based on the RMSE value returned by the train_test_model function and writes the summary with the hyperparameters and final accuracy(RMSE) in logs.\n\n\n\n\n\n\n\n\nTraining The Model \n\n\n\nWe will now train the model for each combination of the hyperparameters.\n\n\n\n\n\n\n\n\nLaunching TensorBoard\n\n\n\nIt's time to launch TensorBoard. Use the following commands to launch tensorboard.\n\n\n\n%load_ext tensorboard\n\n\n\n%tensorboard --logdir logs\/hparam_tuning\n\n\n\nTable View\n\n\n\nOnce it is launched, you will see a beautiful dashboard. Click on the HPARAMS tab to see the hyperparameter logs.\n\n\n\n\n\n\n\nIn Table View all the hyperparameter combinations and the respective accuracy will be displayed in a beautiful table as shown below.\n\n\n\n\n\n\n\nThe left side of the dashboard provides a number of filtering capabilities such as sorting based on the metric, filtering based on specific type or value of hyperparameter, filtering based on status etc.\n\n\n\nParallel Coordinates View\n\n\n\nThe Parallel Coordinates View shows each run as a line going through an axis for each hyperparameter and metric. The interactive plot allows us to mark a region which will highlight only the runs that pass through it. The units if each hyperparameter can also be changed between linear, logarithmic and quantile values.\n\n\n\nThis is extremely useful in understanding the relationships between the hyperparameters.\u00a0\n\n\n\n\n\n\n\nWe can select the optimum hyperparameters just by selecting the least RMSE as shown below:\n\n\n\n\n\n\n\nScatter Plot Matrix View\n\n\n\nThe Scatter Plot View plots each of the hyperparameter and the given metric against the metric.\n\n\n\nThis helps us understand how different values of each parameter correlates to the metric.\n\n\n\n\n\n\n\nComplete Code\n\n\n\n\n\n\n\nThus for complex networks, TensorBoard can give us valuable insights to optimize the model for better performance and accuracy.